Courtesy of Historic VR
Spotlights
August 30, 2022
Medieval life at Guildford Castle comes to VR
Explore the historic landmark's architecture, sights, and colors as they existed hundreds of years ago, courtesy of RealityScan.
Guildford Castle VR is an entertaining and educational historical experience that takes you through space and time to explore one of England’s earliest castles. The experience is available on Steam as a free-roaming, narrative VR experience.
To learn more about the Guildford Castle VR, we spoke to a representative of the creators of this project, Historic VR, a real-time VR studio focused on history, heritage, and traditional visual arts.
To learn more about the Guildford Castle VR, we spoke to a representative of the creators of this project, Historic VR, a real-time VR studio focused on history, heritage, and traditional visual arts.
Courtesy of Historic VR
How did you get the idea to create this VR project?
The idea for the project came from experiencing many VR cultural titles and realizing the vast potential of this new medium to experience history and culture in an entirely new and immersive way. While we found many excellent experiences, we were always left feeling that we wanted more, both in the level of immersion and the richness of the historical information.
Our sole focus was to create a more comprehensive and engaging experience than before, combining a high level of virtual environment realism, meticulously researched historical reconstructions, and a rich audio-visual narrative. We set ourselves the task of creating a virtual experience which would rival—and in some respects exceed—a real life visit to the site.
Courtesy of Historic VR
Tell us about the project workflow.
Our first focus was on capturing and recreating the modern-day environment. We used a FARO Focus S350 LiDAR scanner to scan the overall site and processed the model in RealityScan, providing us with an instant “white box” of millimeter-level accuracy. Throughout the creation of the modern-day environment, this served as a reference for exact placement and shape for all assets—large structures, props, and even the foliage.
Courtesy of Historic VR
We set out with an uncompromising approach to the data capture. Inevitably, VR hardware limitations prohibit us from getting anywhere near the source quality, however we were keen to see how far we could push the quality of the models while still being optimized for VR. This meant taking a large number of high-resolution photographs—over 16,000 photographs in total for the castle keep, other structural assets, and environment props, as well as over 100 LiDAR scans across the entire site.
For the keep (both inside and out), we relied on combining LiDAR with the photogrammetry data, having first registered the LiDAR scans in FARO SCENE 3D Point Cloud software before processing combined data sets in RealityScan. Being able to combine LiDAR and photogrammetry is important as the models become bigger and more complex, and so this is an essential feature for us.
We are specifically focused on using photogrammetry to create extremely realistic, highly optimized assets which can be placed in dynamic game engine environments using a full range of lighting scenarios. This brings with it many technical challenges, and often makes the pipelines complex.
The foundation is high-quality, clean data capture. Because we are ultimately looking for “delit” assets, we always seek to capture photogrammetry data in flat, even lighting conditions. When dealing with very large outdoor objects in variable lighting and under time constraints, data is rarely perfect, so there is often post-processing work involved.
After data capture, our workflow is as follows:
Photogrammetry is such an incredible tool for creating photorealistic assets that it creates a problem—it makes everything else look bad!
We had originally set out to follow more traditional game workflows for the props and other assets, but quickly found that the amount of time, effort and skill it takes to bring them to a comparable quality level as the main photogrammetry elements was infeasible.
We therefore used the same capture methods for all other elements, but using a more optimized workflow:
Tell us about the photogrammetry process.
We are specifically focused on using photogrammetry to create extremely realistic, highly optimized assets which can be placed in dynamic game engine environments using a full range of lighting scenarios. This brings with it many technical challenges, and often makes the pipelines complex.
The foundation is high-quality, clean data capture. Because we are ultimately looking for “delit” assets, we always seek to capture photogrammetry data in flat, even lighting conditions. When dealing with very large outdoor objects in variable lighting and under time constraints, data is rarely perfect, so there is often post-processing work involved.
What's your workflow like after data capture?
After data capture, our workflow is as follows:
- Color correction and pre-processing on raw photos in Lightroom, and output as either TIFFs or JPEGs (depending on the asset).
- Import processed photos to RealityScan where they are aligned, meshed, and textured. (For very large-scale assets, we always use a component workflow)
- Models cleaned in RealityScan before exporting.
- Further geometry cleanup in ZBrush.
- Retopology and UV remapping.
- Texture de-lighting.
- Texture clean-up.
- Optimization using a custom-built pipeline.
What was the process for creating props and environment assets?
Photogrammetry is such an incredible tool for creating photorealistic assets that it creates a problem—it makes everything else look bad!
We had originally set out to follow more traditional game workflows for the props and other assets, but quickly found that the amount of time, effort and skill it takes to bring them to a comparable quality level as the main photogrammetry elements was infeasible.
We therefore used the same capture methods for all other elements, but using a more optimized workflow:
- Mesh cleaned up in ZBrush, followed by creation of a low-poly version in either 3ds Max or ZBrush.
- For hard surface props, such as the gate in the pictures, we tend to use 3ds Max to create very low-poly geometry from scratch.
- UVs created and optimized for getting the highest quality textures.
- Texture reprojection in RealityScan from the master model.
- Texture processing in Substance Painter, and hand-painted roughness and metallic maps.
Courtesy of Historic VR
How did you recreate the massive amounts of foliage?
One of the greatest challenges of the project was recreating the foliage in the modern-day gardens, which are very densely planted and renowned for their elaborate flower displays.
As with the props and other assets, we had to reach a quality level which was in harmony with the photorealistic hard-surface assets, yet which was still within the stringent VR performance limits.
Courtesy of Historic VR
This turned into a monumental task: it was almost impossible to find high enough quality source data for most of the foliage, so we bit the bullet and captured the data for over 60 species ourselves, including more than 20 unique flowers.
Courtesy of Historic VR
For many of the key trees, we used a combination of photogrammetry, LiDAR data, and our own leaf source images to get as close as possible to the actual trees in the gardens.
Courtesy of Historic VR
Tell us more about the entire medieval town rebuilt in 3D.
The recreation of the medieval town started with the castle and palace complex, of which there is a physical model on the ground floor of the castle. When we were scanning this room we realized how useful this could be, so we took the opportunity to do a quick-fire capture (using a spare Sony compact camera we had to hand). Eventually we needed to amend the model based on updated archaeological reports and advice from our historian, but as a first basis, it proved extremely useful.
To ensure that our 3D models conformed to the real world, we used the LiDAR data of the surviving structures to accurately map the palace complex, including the castle arch built in 1256 (which still stands perfectly intact today) and a number of original outer wall sections.
Courtesy of Historic VR
The landscape was created by using airborne LiDAR data with elements resculpted to fit the medieval period based on historical research (for example, moving the course of the river and removing modern urban features).
The creation of the wider town was a huge task, with all the architectural elements—churches, town houses, mills, bridges—modeled from scratch based on archaeological reports and other historical sources. While we could have taken many shortcuts and ended up with a very similar-looking environment, we were keen to be as historically accurate as possible so our users can be confident that what they experience is a faithful representation of the past in parallel to the realism of the modern level.
To walk in the keep today, or any other castle or medieval building in England, you will see only plain, unadorned stonework. This gives the completely opposite picture of how these places would have originally looked—medieval interiors were rich, intricate, and extremely colorful. This was therefore one of the most enjoyable parts of the project and demonstrates the central idea of the experience: to see a radically different past appear before your eyes.
As with the recreation of the medieval town, this involved meticulous research. We took inspiration from English Heritage’s excellent real-life restoration of Dover Castle’s great tower of the same period, though were also careful to independently research each and every element.
The items are all based on surviving contemporary or near-contemporary examples where possible. Inspiration was also taken from illustrated period manuscripts, for example The Canterbury Tales and Eadwine Psalter.
The creation of the wider town was a huge task, with all the architectural elements—churches, town houses, mills, bridges—modeled from scratch based on archaeological reports and other historical sources. While we could have taken many shortcuts and ended up with a very similar-looking environment, we were keen to be as historically accurate as possible so our users can be confident that what they experience is a faithful representation of the past in parallel to the realism of the modern level.
How did you manage to recreate 12th century interior and furniture?
To walk in the keep today, or any other castle or medieval building in England, you will see only plain, unadorned stonework. This gives the completely opposite picture of how these places would have originally looked—medieval interiors were rich, intricate, and extremely colorful. This was therefore one of the most enjoyable parts of the project and demonstrates the central idea of the experience: to see a radically different past appear before your eyes.
As with the recreation of the medieval town, this involved meticulous research. We took inspiration from English Heritage’s excellent real-life restoration of Dover Castle’s great tower of the same period, though were also careful to independently research each and every element.
The items are all based on surviving contemporary or near-contemporary examples where possible. Inspiration was also taken from illustrated period manuscripts, for example The Canterbury Tales and Eadwine Psalter.
Courtesy of Historic VR
What are your future plans for the project? Are you planning on adding more VR interactions?
Our goal with this first project was to get the basics right by developing all the environment art workflows, historical reconstructions, and basic functionality. Following on from the successful completion of Guildford Castle VR, we’re currently looking to build on this core idea and to tackle much larger-scale sites, where we are now confident that we can deliver even higher quality and more optimized environments for a broader range of headsets.
We certainly do intend to add more VR interactions, and this has been a major internal development focus since the Steam launch. In subsequent launches we plan to extend the richness of the passive narrative elements, and also add new, active VR interactions which further engage users. A big priority is to add multi-language functionality to bring the full historical experience to our global, non-English speaking users.
Download RealityScan
RealityScan is free to use for students, educators, and individuals and companies with an annual gross revenue below $1 million USD.Above the $1M threshold? Visit our licensing page to find out about your purchasing options.
Download the launcher
Before you can install and run RealityScan, you’ll need to download and install the Epic Games launcher.
Install Epic Games Launcher
Once downloaded and installed, open the launcher and create or log in to your Epic Games account.
Having trouble? Get support, or restart your Epic Games launcher download in Step 1.
Install RealityScan
Once logged in, navigate to the RealityScan tab in the Unreal Engine section and click the install button to download the most recent version.