Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#1
Hi everyone,

I know there has been some discussion in past of people trying to work around the 8kmx8km map size limit. For me, I was hoping to have huge chunks of forest that the player has to navigate through by roads or hiking, and I think having this huge sense of scale is crucial to making the game feel realistic and immersive.

I've been experimenting with just placing large objects of exported sattelite height data and then using raycasting to procedurally place vegetation. So far this suprisingly seems to work! I have noticed one bug, when I spawn outside of map bounds my weapon disappears, but when I toggle 3rd person and back 1st person I can see it again. Otherwise, the flowgraph scripts that I use to place objects and control AI seem to work just fine. I've actually decided not to use the AI Navigation maps also so I don't need those anyways.

I wasn't sure if this is a viable option and I should continue to just place objects outside of the map bounds? Or, am I in for a nightmare of potential issues and shouldn't continue using this method?
FYI I have only tested this in game editor, I haven't exported the game yet.

Pics below hopefully better show the scale of what I'm hoping to achieve. Thanks for any advice!

Yellow Border= 8kmx8km Normal Map size, Green border = Extended map size outside of bounds
Image
Physics and raycasting to place objects seem to be working OK
Image

Re: Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#2
Hi dinoboy,
Over the years I've developed a simulator based on CE3 which effectively creates a map the size of the world.
Earlier versions were more limited, with maps hundreds of kilometers across.
Terrain was built from brushes roughly 20 km across, using shuttle SRTM terrain height data and satellite images.
I use a very useful program called Universal Maps Downloader to download satellite images for a specified location. It's free for low resolution images
but requires a 50 dollar licence for usable resolutions.
Terrain is built from smaller terrain grids placed together - this is essential as Cry Engine can't display brushes more than 10km or so across.

It started as a flight simulator but I now refer to it as a "simulator", because it has full support for these vehicle types: aircraft, ships, trains, ground vehicles (e.g. cars and tanks) and spacecraft.
The spaceflight system effectively expands the map size to the entire solar system.

The simulator now has a system I call the "world system" - the map is effectively the size of the world. You could fly a 747 from Heathrow in the UK to the US - assuming you didn't die from boredom as you navigate the Atlantic!
It is completely real-world. It uses landclass data to decide which kind of terrain to display. Terrain is built from terrain grids (brushes) sized 14km * 7km (near the maximum size CE3 can display - my tests suggest CE5 can display grids only about half the size).
There are four kinds of objects:
1. Autogen, which can place tens of thousands of objects according to the landclass data - useful for objects such as trees and buildings.
2. Group file objects. This allows detailed local scenes to be built e.g. a fishing village or a city. The code then loads the objects in-game, again depending on the landclass data.
3. Roads. Roads automatically have road traffic.
4. Rail tracks. Fully functional trains are randomly spawned on rail tracks.
5. AI vehicles. For example, if the scenery grid includes an airport it can have any number of AI aircraft which will taxi, take off and land again. The AI vehicle saved game files can create complex scenarios including aerial or tank battles.

With standard settings, the terrain forms a square 140km across. But, crucially, it is dynamic. As you move along, any terrain grids that fall behind you are deleted. Free grids are then placed ahead of you as required. The result is that you are always surrounded by a 170km square of terrain (and objects) irrespective of where you are going.

Cry Engine uses low precision floating point for many applications, so, as you move away from the origin your view may start to shake, particularly if you're in a vehicle. At 100km from the origin the shaking may be very bad (at least in CE3, I'm not sure about CE5). The simulator world system solves this problem, because it keeps the player location no more than 5km from the Sandbox origin.
When the player moves more than 5km from the origin *everything* in the map is moved back. Amazingly, this is essentially invisible, you can only tell when a jump has occurred by watching the diagnostics.

The terrain can be generic or real-world. Just in front of the Harvard you can see the runway of Lukla Airport in the Himalayas - probably the most dangerous airport in the world!

The simulator runs on Cry Engine 3. It's pretty easy to install and use CE3. Currently I have no plans to update it for CE5, as CE5 has serious problems.
If you're interested I'd be delighted to discuss it in more detail.
Best regards,
Chris
Attachments
ScreenShot3993.jpg
ScreenShot3993.jpg (655.58 KiB) Viewed 5466 times
ScreenShot3913.jpg
ScreenShot3913.jpg (516.02 KiB) Viewed 5466 times
ScreenShot3863.jpg
ScreenShot3863.jpg (465.53 KiB) Viewed 5466 times

Re: Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#4
Hi dinoboy,
Nice to hear from you, even after so long (only joking!).

Yes, I'm still developing the simulator and I've made a lot of progress since last we spoke.
It covers the entire planet. The main database is a high resolution bitmap which is a map of the earth. Each pixel represents a 14km * 7km terrain grid, and the colour index gives the landclass for that grid. For example values of 50,51,52.... are "green fields", 80,81,82... are towns and 90,91,92... are cities. Values of 100 and above are reserved for specific scenery areas (not generic). For example landclass 100 is Heathrow Airport.
I used global data lists that list all cities and airports in the world and wrote code to generate the bitmap image from that.
For example, if the landclass for a terrain grid is 50, the code looks up the data files for type 50, most importantly the terrain grid brush model for 50 (size 14km * 7km). If there are other asset files for type 50 the code will load those too. For example, if there is a road data file it will load the road and also it will load the landclass file that defines the objects that will be placed along the road. The code will also load AI ground vehicles that will drive along the road.

"I've been experimenting with just placing large objects of exported sattelite height data and then using raycasting to procedurally place vegetation."
I use the sme method. When placing brush objects, effects and vehicles I use the Lua raycast command to get the terrain height at that point.

I've recently greatly improved the city environment - but it still needs lots of work. In the screenshot the roads, kerbs, pavements, traffic lights, buildings and vehicles are all loaded by the code. As you drive along objects that fall behind are deleted and new objects created ahead. This allows roads to be very long. The longest roads I use are up to 100km long, but they could be much longer. I think driving fast cars is probably the most fun part of the simulator! Incidentally, the simulator has an editor which can be used to create and edit roads and railtracks.

If you have any more questions or comments, please let me know. I'll see if I can put up a short demo video in the next couple of days.
All the best,
Chris
Attachments
ScreenShot4059.jpg
ScreenShot4059.jpg (352.84 KiB) Viewed 3986 times
ScreenShot4073.jpg
ScreenShot4073.jpg (399.5 KiB) Viewed 3986 times
ScreenShot4064.jpg
ScreenShot4064.jpg (601.63 KiB) Viewed 3986 times

Re: Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#5
Hi dinoboy,
Nice to hear from you, even after so long (only joking!).

Yes, I'm still developing the simulator and I've made a lot of progress since last we spoke.
It covers the entire planet. The main database is a high resolution bitmap which is a map of the earth. Each pixel represents a 14km * 7km terrain grid, and the colour index gives the landclass for that grid. For example values of 50,51,52.... are "green fields", 80,81,82... are towns and 90,91,92... are cities. Values of 100 and above are reserved for specific scenery areas (not generic). For example landclass 100 is Heathrow Airport.
I used global data lists that list all cities and airports in the world and wrote code to generate the bitmap image from that.
For example, if the landclass for a terrain grid is 50, the code looks up the data files for type 50, most importantly the terrain grid brush model for 50 (size 14km * 7km). If there are other asset files for type 50 the code will load those too. For example, if there is a road data file it will load the road and also it will load the landclass file that defines the objects that will be placed along the road. The code will also load AI ground vehicles that will drive along the road.

"I've been experimenting with just placing large objects of exported sattelite height data and then using raycasting to procedurally place vegetation."
I use the sme method. When placing brush objects, effects and vehicles I use the Lua raycast command to get the terrain height at that point.

I've recently greatly improved the city environment - but it still needs lots of work. In the screenshot the roads, kerbs, pavements, traffic lights, buildings and vehicles are all loaded by the code. As you drive along objects that fall behind are deleted and new objects created ahead. This allows roads to be very long. The longest roads I use are up to 100km long, but they could be much longer. I think driving fast cars is probably the most fun part of the simulator! Incidentally, the simulator has an editor which can be used to create and edit roads and railtracks.

If you have any more questions or comments, please let me know. I'll see if I can put up a short demo video in the next couple of days.
All the best,
Chris
Hi Chris,

So sorry for late response again! I thought I would get a notification if there was a new response to this thread but not the case. I'll just remember to keep checking back here frequently to stay in touch. I Can't imagine how big of a file that bitmap is for the entire Earth! What language did you use to generate the city pixels from the databases? I'm starting a Masters in CS program this January, but actually don't have a lot of raw experience programming since I have a BS in Mech Engineering. I am fairly skilled at C but haven't had a ton of use for it yet with CryEngine. Python and Blender have been super useful, I've made a few scripts to go through and chop up my heightmap into small chunks, name the new objects, assign correct material, and export.

I'm also shocked you can have roadways 100km, I'm currently just using the spline distributor to create roadways but it can be a bit buggy working outside of the normal map bounds (e.g. I have to reload geometry often otherwise the road exists physically, but cannot be seen. LODs also fail to work outside of the normal map). To get some variation in vegetation, my terrain .cgfs is now has pieces cut out that align to the green terrain. These pieces are then given a different material type, so when the raycast hits and checks material type it knows whether or not to drop forest objects or grassy objects. Unfortunately, I think it would be too time consuming for me to use actual Sattelite terrain imagery and then cut out corresponding objects for the fields. My sattelite imagery repeats every ~4km, so you do have the same pattern of grass/forest/grass/forest occurring. I try to break this up by placing water bodies or large objects for farm fields to make it seem less repetitive. Pic below hopefully better shows what I'm trying to explain.

Sorry if I'm misunderstanding, but it sounds like by your method of moving height field objects around keeps the player inbounds of the map? If so, does this mean you are using normal conventions for AI (like AI Navigation Areas, AI Sequences etc..) Since most of my map is out of bounds I just FG scripts to play animations for characters Cconsidering most are low thinking Zombies, it actually works just fine haha, but it would be nice and probably easier on the engine to work in bounds so I limit the human interactions a bit outside the main map.

If you ever have time to make tutorials or even just show current gameplay footage I would absolutely be interested in taking a look!

Thanks so much for all the advice!

-Andrew

Image

Re: Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#6
Hi Andrew,
Great to hear from you again - hopefully quicker next time!
I've attached the world bitmap. Each pixel represents a terrain grid 14km * 7km. The world's surface consists of 2784 * 2784 grids, so that's the resolution of the bitmap. The file size is 7.39 Mb, so it's not too large.
I used three data sources to create the bitmap:
1. Landclass data.
2. A list of cities
3. A list of airports
I used Lua code to process and combine the data into the final bitmap. The landclass data was the best I could find for my purpose but it still wasn't very good.
For the simulator I use two languages: Visual Basic and Lua. I used VB to create the Aircraft Factory program and also to make a terrain processing tool which converts shuttle SRTM terrain data into .obj files which I can import into Softimage. Aircraft Factory automatically creates about 6000 lines of Lua code for each vehicle type. If I update the vehicle code I update the appropriate library files used by AF. I can then use batch mode to update the code for all the vehicles in a few seconds. Without AF updating the code for all the vehicle types would be a nightmare!
As you probably know, Lua is the scripting language of Cry Engine. It's very powerful and easy to use, the syntax is vaguely similar to Basic. It can easily access a large library of C++ Cry Engine libraries. E.g. to set an entity position, the code is self:SetPos(x, y, z); To get the terrain height by ray trace it's
local hits = Physics.RayWorldIntersection(EntPos, {x=0, y=0, z=-8000}, 1, ent_all, self.id, self.id, hitData);
Crytek keep threatening to end support for Lua, but it can still be used in CE5, though possibly it requires GameSDK. However, I'm not developing the simulator for CE5 as CE3 is far more suitable and CE5 seems to have a fatal problem, possibly two.

Roads and railtracks are created by the same Lua entity. Each road or railtrack is a section 30 meters long which can have a set bend angle e.g. zero or 15 degrees. The entity deletes sections beyond a set distance and loads sections that have come into range, so very long roads work fine as only a sub section are loaded at any time. As I think I mentioned, I've used road lengths up to 100km, but there's no reason why roads much longer shouldn't work. I might try e.g. a 1000km road to see how it works.
It's a lot of fun using the simulator editor to create and edit roads or railtracks. It works in two ways. You can select individual sections and change the bend angle. Each time you do that all the subsequent sections will be re-drawn. The second method uses waypoints. Each waypoint is drawn as a sphere. You can select a waypoint and move it as required. The road follows the waypoints just like a spline following its control points. One big benefit is that you can edit a section of road without having to re-edit all the following sections.
I recently updated the roads code so they can have pavements and curbs, this works very well for cities. The code also allows objects to be placed along roads e.g. traffic lights shown in the previous post.

"so when the raycast hits and checks material type it knows whether or not to drop forest objects or grassy objects."
That's a nice idea. I have toyed with the idea of creating code that can analyze terrain textures and recognize buildings and trees. Currently the editor allows you to choose from a library to place and move objects on the terrain. This allows realistic alignment e.g. teees with tree images on the terrain, but of course it's very time consuming. To be able to automate this would be magic.

Yes, the world system literally moves all elements of the map dynamically in-game. This keeps the player withon 5 km of the Sandbox origin, even after travelling thousands of miles. I'm actually amazed it works so well. The jumps are essentially invisible. However, there is a catch: all elements in the map must be capable of being moved dynamically and accurately whenever a jump occurs. This works fine for terrain grids, all brush objects attached to the grids, all simulator vehicles, the sky dome, clouds etc. But there's one exception: effects. It seems that, once emitted, the effect particles can't by moved by the code, even if the effect itself is loaded into an entity slot. So, if I'm flying a fast jet and I fire a missile, the smoke trail will be fine until a jump occurs. After a jump the smoke will be displaced by 5 km, which isn't very realistic.

Possibly the code could move standard game characters in a jump. But it's not a problem as I don't use standard game characters, including the player. When you're walking in the simulator in game mode, you are actually in a vehicle named, appropriately enough, Walker. It is very similar to the standard player character, and has some enhancements e.g. commands to change the view height and walk speed. Walker is actually a standard simulator vehicle, the main difference being that it moves on legs rather than wheels. Because it's a simulator vehicle, it's jump-friendly. Walker can also be used for AI characters, including animals. They can be set up to follow waypoint paths just like vehicles. They should also be capable of combat just like vehicles, though I haven't tried that.
So, in a nutshell, the world system does keep the player in the map bounds, but all elements of the map must be jump-friendly. Things like AI navigation areas probably wouldn't work, but the simulator does have alternative methods.

I saw your videos showing the transition from space to a planetary surface. I've spent a *huge* amount of time on this problem! The spaceflight system works really well, you can easily fly from Earth orbit to Saturn. But the really big problem was to make a realistic transition between space and planetary surfaces. I've done a lot of work on this recently so that now the world and space systems work well together. The transition uses smooth control of the transparency of the planet sphere. In orbit the planet is fully visible while the terrain is disabled. As you descend to the surface the planet becomes more transparent and finally fades away revealing the detailed terrain below. There probably isn't a perfect solution with Cry Engine, but this works pretty well.

I'll see if I can make a demo video in the next couple of days.
Meanwhile, you can see the manual. Although well out of date it contains a lot of information, and there is some info on the world system.
https://filedn.eu/lVP0lntroxNm8qxOBmfu6 ... anual.html
As always, feel free to ask any questions!
All the best,
Chris
Attachments
Landclass_airports_final.jpg
Landclass_airports_final.jpg (1.09 MiB) Viewed 2802 times

Re: Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#7
Thanks as always for all the insight on this huge project! Really like how you opted to create a vehicle simulator as a jump-friendly alternative to the standard player/ai character. I'm still using the typical character method, but honestly for games like ours that aren't story/character driven I feel like using characters with advanced AI, behavior trees, physics etc. is overkill and probably is negatively impacting my game performance. I just need my characters to perform maybe 2 dozen diff animations and literally their AI can be governed by simple flowgraphs.

No other huge updates on my end other than trying to find methods to make my big map feel less repetitive and have unique parts of the landscape that players can recognize and use to help navigate, otherwise they'll feel like they're in an endless void. Fortunately I can create quick designer objects in game and drop them in to represent a farm field, or rocky terrain on a mountain side. Its time consuming but nice to be able to work completely in editor for once.

On a sidenote, have you ever played DayZ or ARMA II/III? I originally got on into map making modding for DayZ and I recall the engine is built around creating huge maps and there didn't seem to be any limitations. Unfortunately the ARMA engine is not open for commercial development, and all the code is written by Czech developers so it makes things a bit complicated, but I still had a lot of fun making a couple maps with their provided dev tools (youtube link below of an example I worked on)

https://www.youtube.com/watch?v=7VP3Z9iYCoc

P.S. I clinked your link just a sec ago and i got 404'd, wasn't sure if you could post again? I def want to check out all the docs about ur development!

Thanks!

-Andrew

Re: Working Outside of Map Bounds to Create a 60kmx60km Mega Map?

#8
Hi Andrew,
Yes, the free demo hosting site timed out. I've now put it on Netify. This should work:
https://musing-gates-cf240a.netlify.app

As I mentioned, I don't use the standard AI characters. Instead AI and the player characters are instances of the Walker vehicle entity. This ensures they are jump-friendly. If you are walking in-game you are actually driving the Walker vehicle - of course, the view panning and controls are the same as the standard game character, but with enhancements. If you walk to a vehicle you use the standard F command to enter it. When you enter a vehicle you leave Walker behind, but the vehicle code lets you walk around the interior in exactly the same way. For example, if you're in the 737 or Constellation you can realistically walk around the cockpit and passenger cabin while flying at hundreds of miles an hour. This is one of the unique features of the simulator. In a similar way you can land your aircraft on a moving aircraft carrier, taxi it and take off again. You can also exit the aircraft. You are automatically attached to the carrier so you can walk around the deck, watch aircraft taking off and landing or even enter and drive a ground vehicle or aircraft.

I've used Walker to create AI animated characters, both human and animal. In a demo map a man gets out of a chair and walks, other men walk, run and pause, there's also a walking chicken (yes, it does cross the road!). Animated character models and animations can be assigned to the Walker entity. Because it's a standard simulator vehicle it can use the saypoint system. A waypoint file defines the x,y.z positions of any number of waypoints. A vehicle can then move from one waypoint to the next, thus defining an AI path. Any waypoint entry can include any number of commands together with parameters. The commands are automatically executed when the vehicle reaches the waypoint. There is a large library of waypoint commands. In one example a car ferry sails in to the dock and stops at a terminal. A group of cars drive up to the terminal and park themselves in pre-determined positions. When the ferry opens its door the cars drive on and park themselves on the ferry. When this is complete the ferry closes the door and sails off. When it returns to the terminal the cars drive off.
All the waypoint commands are listed in the manual. The main emphasis of the simulator is on vehicles, but more waypoint commands aimed at AI characters could easily be developed.

I did try an Arma demo many years ago. It was pretty good but I do remember that I didn't like the method for steering vehicles.
I'll try to make a short demo video this week.
All the best,
Chris
Attachments
ScreenShot3976.jpg
ScreenShot3976.jpg (674.71 KiB) Viewed 607 times
ScreenShot3966.jpg
ScreenShot3966.jpg (470.74 KiB) Viewed 608 times
ScreenShot3964.jpg
ScreenShot3964.jpg (243.75 KiB) Viewed 609 times

Who is online

Users browsing this forum: No registered users and 4 guests

cron