Growing My Own Mountains or the Plugged Off Plugin

March 24th, 2021

Eulora's legacy client requires no less than 12 image files to create just one sector of the whole wide world and this piles on, of course, the megabytes transferred and stored under the generic label of "art assets". The reason for such shameless requirement is the reliance on one of those oh-so-flexible plugins that the engine (CS aka Crystal Space) is plagued with: the terrain plugin that supposedly generates programmatically all sorts, except in practice it still expects 5 detailed, pre-made maps, of which 4 (basemap, lightmap, materialmap, normalmap) are anyway calculated otherwise based on the 5th (the heightmap).

Perhaps you wonder just what does the plugin do with 5 maps anyway. Well, a deeper dig into the code of it all revealed that the plugin just uses them all to create over and over again, at each start of the game, the same terrain mesh - except it never quite calls it directly mesh (it's terrain!) and it creates it according to its own rules and decisions as to what all these maps mean and how they are to be interpreted. Basically the terrain plugin expects that it owns the terrain entirely and so it gets to actually decide on what height is where (according to its own, internal rules, too!) and even whether the player can or can't go somewhere - what representation vs meaning, what client vs server role? In this brave new world of no depth at all where everything that matters is how something looks, the graphical representation thinks itself the master of everything and so gets repeatedly in the way at all stages until the inevitable point where I finally give up on working *with* it at all and therefore ditch it entirely, just as it has happened before and as it will continue happening: if you keep saying you can't do this and can't do that and won't do the other thing either, the result is not that whatever it was to be done doesn't get done - it's simply that it will not be done *with* you but rather *to* you and not in ways of your own choosing either.

Due to the above stupidity of the terrain plugin in CS, what started as an attempt to merely calculate the terrain's final height at any given point, ended up in quite a bigger and more tangled chunk of work to do but the net result is the addition to my gfx pipeline of a direct, full creation from scratch of a terrain mesh that can then be sent to the client to use as it is - no need for sending 5 maps over the wire, nor for any further polygonizations and calculations of height each and every time someone fires up the client: at least if it has to use a rasterised format, let it be calculated just once on the server and then reused forever by all the clients that may ever need it.

As usual, things didn't initially go very smoothly but that was quite expected: first the whole thing didn't show at all1, then the "object is not closed" complaint kept on complaining even on the closest of them closed objects2, then the movement went haywire, whole mountains kept playing hide and seek with my view, bits and pieces started flickering, vanishing, blackening out or any combinations of those three. And then, finally, even the return to the "old way" (aka plugin based) failed, revealing at the least expected moment that the plugin itself was in fact no better at handling the sort of situations I was stressing my poor mesh with.

All the above out of the way and written here once for all the future uses, let's illustrate it, too. First in line and as seen by my aptly (if algorithmically) named Perfectput Dot rabbit, the amazing whole land-vanishing capability of the terrain generated by the plugin - basically the culling itself fails just as much on my mesh as it does on the plugin's creation:

Now you see it...
mesh_terrain_7_640.jpg

Now you don't...
mesh_terrain_9_640.jpg

Set entirely at ease by the above as to failure being a given anyway, I started growing mountains out of the submerged plains, which was simple enough when all it takes is cranking up the maximum height parameter of my generator: the mountain grew and grew but it still remained stubbornely and quite ridiculously pointy:

mesh_terrain_1_640.jpg
mesh_terrain_2_640.jpg
mesh_terrain_3_640.jpg
mesh_terrain_5_640.jpg

While the last image above is at least not *that* pointy, finally, the whole exercise made it quite clear that there was some deeper trouble in there, as indeed there was: first of all, I had an error in my height calculations (so that the factor applied was not at all what I thought it was) and second but more important, it seemed to me rather silly to calculate the full heightmap of 1024 x 1024 size to then pick for use a smaller grid (to keep the size of the mesh file itself reasonably small). Basically the added detail that the heightmap held was just thrown away because the polygonization used *less* detail, not more (if it were more, I'd have interpolated and in principle it should smoothen it further so it's possibly closer to what I'd want, anyway). So I tweaked a bit the code to simply add the actual scale and then generate the heightmap only as big as needed: a scale of 4:1 for instance means that real world unit is 4 for every 1 in the heightmap and therefore a real world map of 1024 x 1024 can be fully obtained with a heightmap of only 256 x 256 (since 1024/4 is 256). Otherwise just using every 4th value in the heightmap doesn't really bring much benefit, as it's not "more detailed" since the detail itself is just glossed over. In any case, this part might still need further tweaking but for an illustration, here's the resulting landscape with max height set to 100, scale factor set to 8:1 (hence generated DS heightmap of only 128 x 128) and values used for DS seeding from -5.0 to 5.0):

mesh_terrain_4_640.jpg

For this initial testing stage, I didn't bother with creating any actual texture for the terrain but I don't really see why that would be any trouble really, whether doing splatting or anything else really - one big advantage of ditching the terrain plugin is that there is now no limit whatsoever on texture or materials, I can even mix as many materials as I want, no "exactly three" restriction anymore. At any rate, the images above simply use the "sand" material (both image and normals, hence a bit more than the plain textures used for my meshes so far) everywhere, with a plain mapping - while the effect in the distance seems reasonable enough to me even with this tiny material, the close up does indeed seem to require more detail. Possibly the current 512 x 512 texture is just stretched too much on the triangles of the terrain when close enough and since I don't do any "lod" shenanigans, it looks for now as illustrated above. Nevertheless, this should be improved simply by using a better texture overall (and since that would be a single one rather than 3, it can even be 3 times bigger without any increase in size of the whole .zip as compared to the current one).

Worth noting also that in all the above illustrations, the water and the sky are unchanged - they are still created entirely client side, with just the texture and the normals sent by the server. I don't see any reason to change this (both sky and water are fixed, regular shapes) but at the same time, having now increased (more than I ever wanted) my hands on experience with messing up meshes, I tend to think it's probably worth changing on clientside the *shapes* in use, especially for the sky (and as a result, possibly, the texture to match, too): the current sphere is fine in a way but it creates this unwanted effect that tilting the camera at any point low enough to the ground results in one looking *under* the ground where there is a whole half sphere of... sky. Since "filling" this half sphere is apparently something just not working (because there is no depth to anything, no matter how defined), the alternative that I can see would be to have the world "container" simply as a cuboid with the terrain resting on the very bottom (so that there isn't anything to look at underneath, no matter what tilting) and then simply add a half-sphere for a dome effect or otherwise, possibly even texture the corners of the ceiling out of sight.

All this opens up of course a whole new set of potential parameters and/or approaches for terrain generation but whatever parameters are chosen in the end, the pipeline still works the same so I'd say that the mechanics are now at least in place and the rest can then be easily tweaked further at any later time. What changes otherwise as a result is the "Outdoors" object in the hierarchy of Euloran objects since it looks set to simply contain a mesh for the terrain and then, perhaps, the materials for terrain, sky and water, unless those end up also given separately perhaps.


  1. While CS docs are clear enough that the "front" or visible face is the one with vertices given clockwise, the bit missing was that CS's z axis points forwards! Noting it here for any future puzzlement on this topic: the "top" (or "far") left corner will therefore have the highest z coordinate and the lowest x as opposed to lowest z and lowest x. The rest then falls easily into place, as one just literally needs to list the vertices in clockwise order. Worth noting also that a closed 3D shape with faces oriented so that they are visible from the outside will therefore have... transparent sides when seen from the inside. This is the illusion of 3D in computer graphics at its most cumbersome: there is literally no depth to any of it! 

  2. Oh boy, how infuriating this was: what the stupid thing checks for is not exactly whether the object is geometrically closed but rather a much more custom interpretation of "closed" namely "does this object have at least 16 triangles and in such case, does it also have all edges - exactly as given - shared between at least 2 triangles?" Specifically, this means that you can't really easily define a 3D shape with different level of details on its different sides because you have to match the number of triangles wherever the sides meet, basically there's no way to split one side into several smaller ones for added detail (even if geometrically the whole side is thus covered). In turn, this brought at first the horror realisation that making the terrain a cuboid as opposed to just a top sheet would then require effectively *doubling* the vertices and the triangles (which is not a negligible doubling when talking of a whole sector of the world). Nevertheless, in true open source spirit, it turned out that *ignoring* the whole complaint entirely is the best course of action - if it whines that even closed shapes are "not closed" then it will get fed fully open shapes since it's basically not worth catering to. Funnily enough, this made otherwise no discernible difference in practice, at least not beyond the presence of the complaint itself in the console. Go figure, it turns out that I am *still* paying too much attention to what is nothing but empty pretense, sheesh.