DDS format ObjectLightMaps = performance boost...

For info on the site, community and 'battlefieldy' things
User avatar
Senshi
Posts: 697
Joined: Sun Oct 18, 2009 1:14 pm
Location: Germany
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Senshi »

Thank you very much. I think that should be a sufficient go-ahead.

Sadly it's a real pain in the ass to have to work with Max 7 with this. How luxurious my new Max D 2012 with multi-core is in comparison...Max 7 is a lagfest with this map. But 1514 objects is probably top-notch anyway... ;)

Just a test render after hours of work on the §$%§ object library (that rescaling stuff was a pain until I got it to work correctly...), I only placed a generic skylight, nothing fancy:
Image
The ground textures still are the old BC42 ones with baked lightmaps, I'll redo them completely via Photoshop.
Setting up the gazillion of lights will be work for tomorrow...I fear it'll require a lot of testing until I get good lighting properties that look "adequately" for the ambient fire (all the boxes remaining on the screen above are either smoke emitters or ambient fires...)... :?

Also, just tried the "Set up" tool in Rexman...The object name doesn't change, but when I go to Render to Texture there's a "element name", causing the output name to be: rusruin02_m1_460-76-547.tga . I guess this is the magic of the key? If so, I think I got everything I need :) .

Do you have recommendations for lightmap sizes? Let's say small means sandbags, barrels, crates, benches etc, medium means sidewalks, trams, trains, car(wreck)s and large means houses. Maybe one setting "Huge" for the huge telemark building and other assorted huge factory structures (the long large building between the two bridges, it has a fully walkable interior and lots of action going on there).

Regarding the effects: Alright, I guess this means it's time again for a good show of trial&error ;) .

EDIT: And obviously AFTER I created the darn object library I noticed this fancy website: http://www.battlefieldsingleplayer.com/ ... ibrary.htm
Here is described a fast and easy method on how to create max object libraries very quickly and with no effort at all. Using Rexman tools.
This one belongs to it as well (tutorial on rexman lightmapping): http://www.battlefieldsingleplayer.com/ ... torial.htm

In general I'd like to remind everyone of http://www.battlefieldsingleplayer.com/ ... ing_Guide/
It's definitely worth a bookmark and answers A LOT of questions around BF42, from the very basics to pretty suprisingly advanced stuff. Also, I downloaded the complete webpage (siterip) as a local backup, as this site is hosted on the unstable planetbattlefield and it probably was just an oversight this was not yet deleted, as most of the other tuts and images on that site :( .

And another EDIT:
Rexman is fooling me :( . If you look at his page and everywhere, the import meshes rollout should look like this:
http://www.rex3d.net/bf/index.htm
For me, it looks different. Quite so, actually. Especially no "import all from folder" option, so no batch import. Batch importing treemeshes works, however...it seems he kicked that function out of the last version of the tools. Does anyone have an OLDER version of the tools?

EDITx: Found an old set of tools, but it only allows batch import of the COL mesh :( which obviously is insufficient.
User avatar
Senshi
Posts: 697
Joined: Sun Oct 18, 2009 1:14 pm
Location: Germany
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Senshi »

Sorry for hijacking this thread like that, but the flow of discussion took us far from topic again, I fear ;) .

Well, not being able to batchimport pissed me off (also, I sure as hell wasn't going to import all 5000+ .sm files of BG42 BY HAND), so I decided to return to the good old motto: If it doesn't work, MAKE it work!
Who can spot the difference? ;)
Image

Which means: I created my own BFMDT update that allows batch importing of .SM files. Yeah. That.
I don't recommend anyone trying to do that, I never worked with Maxscript before and it was a huge trial& error process with looking at dozens of tutorials and other script examples in order to get the right idea how to make this specific thing work...pretty much like everyday BF42 modding, actually.
The good thing: It even reads the other settings so only the files you really want get loaded. However be VERY careful when importing huge piles of .sm. When I tried to load all the objects from BG42, it worked (Max7 froze during importing for several minutes, that is normal). But the second I activated a viewport, 3ds Max had a nervous breakdown and cried at me for being too mean, forcing it to display hundreds of thousands tris and simply shut down.

I'll see to it that I also implement this for Treemeshes, so no need to use the rexman tools for this at all anymore. Plus, I still have to test some cross-settings to make sure this doesn't blow up in my face.

EDIT: Expanded it again. Now if you use "Folder Import" the imported files no longer will be named LOD01_X-1 as before, but just X. This means you can directly use that for library-building for REX-tools, because the object names are correct. This name scheme is ONLY used if you use Folder Import combined with the "Only main VIS" selected. If it's unchecked and you load multiple LODs, using the REX scheme would mean all LODs would have the same name. So I disabled it ;) . COL, shadow etc. meshes will still use the old name scheme as well.
User avatar
fo0k
Posts: 1433
Joined: Fri Oct 16, 2009 4:21 pm
Location: UK

Re: DDS format ObjectLightMaps = performance boost...

Post by fo0k »

great idea :) batch import was always something lacking from the MDT. I want it. :)
User avatar
Senshi
Posts: 697
Joined: Sun Oct 18, 2009 1:14 pm
Location: Germany
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Senshi »

Alright, a first release of it. I was so bold and named it 2.76, following the original DICE revision numbering.

Maybe it should be added to the general BF modding tools list and a new thread opened for feedback/bug reports?

It's not thoroughly tested, so please report any issues and quirks you encounter (especially if Max spits out script errors). It also has only been tested on Max7, I have zero idea how it fares on older (or newer) versions. In theory it should work with Max 5,6,7,8,9. Not the new 2010 upwards.
Also, if you have additional quirks or suggestions on how the BFMDT could be enhanced, go ahead. If it's not too difficult I might just be able to pull it off. I'm still new to all this Maxscript stuff, so don't expect magic (especially no "make it work on Max 2012" kind of magic...).

Features added:
Standardmesh folder import: It is now possible to import all standardmesh files from a whole folder. This works in addition to the other settings, so be sure to select only the object parts you really need (vis, col, shadow etc.) or you get very very many files.
Name-scheme: If a folder is imported and only the settings "Only Main VIS LOD" and "Visible" are selected, the imported objects will be named exactly like the objectname, without the LOD01 prefix and -1 suffix. This is done because this particular combination is almost exclusively needed for building object libraries for use with Rexman tools and 3ds Max level editing, this prevents the tedious renaming work required before.
New fancy logo: Click it and get to the coolest BF42 modding page on the web :) .

Known issue: If importing a folder with a huge amount of .sm, 3ds Max gets killed. This is no script error, 3ds Max simply can't handle thousands of objects simultaneously in a single scene. If you have this issue, split your folder into several smaller subfolders (~500-700 files should be okay, this very much depends on the number of polygons/tris the objects have).
Another limitation: The absolute maximum file number the script can handle is 1116 elements (sm+rs) or 558 .sm files. If there are more, a script error is spat out and the importing is halted after the 558th object (no editor crash!). This simply is a limitation of the script engine in 3ds Max, nothing I can do about that. For the low-poly vanilla BF42 this means you have to split your SM folder into 3 separate ones. For other mods that often use far more high-poly objects, the first crash-issue might be more of an issue.

It also is possible that the importing gets disrupted when the importer hits a file that is "bad". You get an error message saying that sth. with a MAT import failed. This means the .sm you tried to import is garbage and either has to be fixed or removed from the import folder. After this error you have to close the Toolbox and reopen it in order to reinitialize it.
I'm working on making this "silent", so no error popup and importing just continues without the bad file being imported. Next version...

Install instructions: Simply unzip into your 3ds Max folder, accept all overwrites.

See attachment for file.
Attachments
BFMDT_2_76_www.bfmods.com.zip
(392.63 KiB) Downloaded 917 times
User avatar
Apache Thunder
Posts: 1210
Joined: Mon Oct 19, 2009 2:48 am
Location: Levelland Texas, USA
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Apache Thunder »

Very nice! Perhaps those that manage to compile all of the vanilla statics or the statics for the popular mods can post up their max files? :D Hey perhaps you can take a look at the old rexman scripts that has the heightmap/sun import tools and in particular the setup light map pane that adds the element names to the objects in the render menu so they have the correct file name. See if you can integrate those functions into the MDT Scripts. It would allow for most users to not have to carry around two script sets for light mapping. Perhaps add it as whole new section in the main dialog.

As a bonus, once you learn enough of the scripting you can figure out how to fix the scripts to work with Max 9/10? That would be a big bonus as right now I have to import new meshes in Max 7 and make sure my compiled Max file for my objects isn't re-saved in max 10 or I would lose access to them all in Max 7. You could make it as a separate version and maybe remove the animation related stuff until you get that working too since I think the animation and treeMesh import/export stuff may have their own script problems in Max 9/10. I don't know if it can be done in a way that would allow it to still work in Max 5-8, so it will likely have to be a separate version of the scripts. I think once the scripts are updated to work in Max 9/10, it should last quite a while before a new version of Max comes along and brakes them again. :P The MDT scripts is the only reason I haven't stopped using Max 7 entirely... though I think I may still use it for the BF2 scripts which I don't know if they work in Max 10 or not...

Also, I discovered a tidbit of info while tweaking the shadows. The Shadow Bias setting (default is 0.5) found in the Area Shadows settings pane for the Area Shadows engine (and may be available in some form or another on other light engines) affects clipping distance of shadows.

Basically it controls whether or not a mesh piece will cast a shadow on another object or itself based on the proximity of the mesh to the other mesh. For example, you have a store sign that is protruding from a wall. The base of the sign that is attached to the wall will cast a shadow on the wall if the bias setting is set to 0 or a very small number.

But I found that in my BFH maps the default setting does not work well. My signs and other similar pieces of objects will have the parts of the base of the mesh that is protruding not cast a shadow which will result in the shadows looking like they are out of alignment ingame.

See it as a "clipping distance" of sorts like with mesh proximity to cameras. The closer something is to another object, the bias setting comes into play and then tells it not to cast a shadow on the portions of a mesh that is too close to the other mesh that the shadow is being cast on.

So if you are having a problem like this in your max lightmapping projects, this is a heads up. Try reducing the bias to 0.1 at the least. It can be set to 0, but I guess a tad bit like 0.05 or something is still needed for reasons I have yet to see. :P

This may also solve issues with light "leaking" through wall corners in indoor scenes and such (areas where a mesh intersects another). I think I noticed this issue with my Berlin night map at one point but could not find out why it happened. Well I think I finally found out why and this setting is the culprit. You may be safe off just setting it to zero. At this point I can't think of a reason not to set it to zero...

Here's before and after to show the effect:

Before(Bias set to 0.5):
Image

After:(Bias set to 0)
Image

It also effects the windows and the bars over the doors as well so this is definitely something that needs to be checked prior to the final texture rendering. :D

Also a tip for working with the hieghtmap. I no longer have max output all the individual TX files. I simply kill the materials for the heightmap (setting them all to 1 since each piece will have their own material for each texture) then do a UVMap on the mesh so that I can use a single 8192x8192 texture file for the hieghtmap so that the entire terrain gets lightmapped at once. I then split the resulting file with the TGA splitter tool. The main advantage is I can make quick adjustments to the entire file (especially helpfull for when I go to make a shadowlightbits friendly version that has only the shadowed areas in B&W which I then have BF1942 merge with all white terrain textures. This allows for better lightmapshadowbits.lsb files in the end. At least for my mod anyway. :P)

You may or may not need to do a flip on the vertical axis on tiling settings for the UVmap if your compiled terrain texture appears upside down on the mesh. Just use the default Planer option for the UVMap. Aside from the tile flip on vertical, you don't need to do much to it after adding the UVMap modifier. After your done with the changes, right click the modifier and select Collapse All. This removes the modifier and makes the changes permanent for the scene.

EDIT:

I am thinking of redoing the lightmaps for my Berlin Night map. Any one interested in wanting to play the map? I want to fix the leaking issues mentioned above + the map is really old and I have learned quite a bit of new lighting techniques since I made the map. The map was made prior to me learning how to use skylights. Which is one thing I think will improve this map greatly. :P

Also another great modding idea! Once your done lightmapping the map you can use the top down view in max and do a max render of it and use that as your minimap in higher resolutions then what BC42 or ED42 can provide! Plus it would show off the map with the proper lighting and such. ;) Press F10 in max to bring up the render dialog. This one is different from the bake to texture dialog you use to output lightmap textures. I use this dialog a lot to test view my lightmapping prior to final render time. This dialog will render a screenshot of your scene from the selected viewport and current camera position in the scene. The Top-Down viewport is the best option for making minimaps. Just zoom out until you can see the entire map (make sure to disable wireframe view and make sure the textures are enabled and show up in your max scene) Then output as a high res image and use Photoshop or similar app to crop out the areas outside the map and resize to a standard resolution accepted by BF1942 like 1024x1024 or 2048x2048 which are the usual resolutions for decent minimaps.

This will only be helpful for you if you bothered to get most of the object and terrain textures to show up in Max. If you set your max paths to your extracted game files in the correct manner, it should not be too much trouble for most mods.
ImageImageImage
I have cameras in your head!
User avatar
fo0k
Posts: 1433
Joined: Fri Oct 16, 2009 4:21 pm
Location: UK

Re: DDS format ObjectLightMaps = performance boost...

Post by fo0k »

http://apatchwork.ytmnd.com/

I intend to make some kind of epic map by christmas.. just finding the moment. All these tips building up to make the most of it, thx!


and edit: just noticed the download link Senshi.. will look forward to trying it!
User avatar
Senshi
Posts: 697
Joined: Sun Oct 18, 2009 1:14 pm
Location: Germany
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Senshi »

After I posted this stuff on BG42, Haribo came out of nowhere and posted a link to MaxScripts that are supposed to work with Max8,9 and 10.

I have NOT full tested them yet, the only thing I can confirm so far is that importing .sm in 3ds Max Design 2012 works with them. So I'm fairly certain they'll work for the lower versions as well. I have not tested other functions yet. But that itself is pretty sweet.

And no, I have absolutely zero idea where he found that version, I'm still trying to get that info.

Again, thank you for the elaborate answer, Apache. Some helpful tips of experience there that would probably have caused a lot of nuisance and searching to track down.
I have fiddled with my map already and it's coming off nicely so far, despite 3ds Max getting laggy as hell with the 674.308 polys on the map...I transfered the whole map thing to 3DS Max 2012 in order to get it to work at all (plus the very helpful new feature of "real-time-lighting" in viewports. Helps A LOT when aligning and positioning lights). Working that map with Max7 simply does not work, it crashes all the time.

I'll see that I can get Rexman tools en par as well, I'm not sure what you mean in your first paragrahp, though. You want the Rexman features incorporated into MDT? Meaning the Import objects position, import hieghtmap, Import sun, "Setup Lightmaps selected"? I can not promise anything on that front, as they are both utterly separated and combining two foreign scripts might prove "a bit" of work...but yes, I'll work on it, because allowing use of all the tools in newer Max series just would be so insanely awesome (and be absolutely necessary for me to able to export my map, actually...).

EDIT: Just noticed that all the Rexman tools seem to work absolutely flawless in 3ds Max 2012. This is very helpful. If you can confirm the attached Max8_plugins to be working, I'd restart my scriptwork based on them, adding the batch import stuff to it and trying to link the Rexman features in as well. Considering that they work already, just linking them in shouldn't be too difficult (I hope...).

EDIT2: Well. Pushing the BFMDT over to 3DS 2012 (sorry, I only have 7 and 12, can't test in-between-versions) was surprisingly easy. Only a minor handful of code thingies that have changed, and altering those was really easy thank to the very good "What has changed in MaxScript xxx" documentation provided by Autodesk, easily allowing to deduct what functions have changed and what the correct replacement is. Now the MDT "boots up" without errors, that is.
Still has some issues, obviously, currently I can't get textures to be imported along with the sm. The BF_RS-Shader is applied fine, but with "None"-Bitmap selected. Once you select a texture manually in the mat manager it does display as intended and everything is alright. But that is something the MDT really should be able to do by itself. I'll also have to thoroughly test the other functions of the MDT to see if really everything is as green as it looks now. Once all is clear on the shore, I'll pitch in the Rex Tools and ...oh no...I'll have created a MONSTER! ;)

EDIT3: Another update. Things are far more problematic down the road, it seems. No idea how to tackle the RS_shader problem yet, especially a I have the 64bit version of Max12 and the "old" plugins are 32bit, thus not accepted by Max12. Main problem is that the max8_plugins I attached are from an old version of the MDT (2.5 I think) and thus lack a lot of the newer features (all of the BFV support, for example). Also, about 60% of the code has changed between them, making "comparing" them pretty difficult, if not useless. I've tried going through the RS Import script, but man, that's a whole lot of stuff in there. That will take some while. Also the .ske and .baf tools are not working correctly, there's multiple issues on that front, so I pushed that on the back shelf. Same goes for the "Vehicles"-Import/Export. Lots of things changed and a very complex script. I'll focus on the RS_Shader and the Rex tools first, that should be sufficient for a first release to the masses I think.
Attachments
max8_plugins.zip
(251.39 KiB) Downloaded 897 times
User avatar
Apache Thunder
Posts: 1210
Joined: Mon Oct 19, 2009 2:48 am
Location: Levelland Texas, USA
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Apache Thunder »

Very cool. Yeah I noticed the textures not loading in issue with the max 8 version of scripts that I have. I don't recall my copy of max 8 scripts ever working in Max 9/10, so maybe yours are different somehow. Thanks for the hardwork on the scripts. :D
ImageImageImage
I have cameras in your head!
User avatar
fo0k
Posts: 1433
Joined: Fri Oct 16, 2009 4:21 pm
Location: UK

Re: DDS format ObjectLightMaps = performance boost...

Post by fo0k »

I would like an app like this...

when you have 5 mins.. :)

Image
User avatar
Senshi
Posts: 697
Joined: Sun Oct 18, 2009 1:14 pm
Location: Germany
Contact:

Re: DDS format ObjectLightMaps = performance boost...

Post by Senshi »

You been at the funny cookies again, fook? :P

Progress report: Beat down the .rs-Shader-Parser so it at least imports materials correctly now (and thus assigns textures as well if an extracted texture folder is specified as "external path" in 3ds Max. No, I won't include auto-unrfa-ing of textures needed.
Everything else is still ugly and broken. Also, GMAX-support has been dropped (eh, pick the old tools, guys, totally sufficient for that) and BFV-support is uncertain. I'm cutting down on "things that should work" currently and will be very happy if I just get the .sm/.rs parsers and import/export scripts running. Those are tricky as hell as is...SM exporting currently is not available and will require some serious work as well. The same goes for skeletons (limited), animations (FUBAR) and vehicles (not working). Vehicles probably will also completely dropped or changed, as the current approach with extracting and writing data directly from/to the game .rfa is complex and I actually don't like it. I want the tools only to work on my fancily stored extracted files.
Goal for first version: SM Import working (incl. .rs), Batch import. Rexman tools. SM Export as bonus, hopefully.

I mean, editing the "fancies" and the tool GUI itself is very straightforward and easy, but the background functions like the parsers are insane. Funny note: Most of the background functions of the MDT have been taken by DICE unaltered from Rexman and just included into some smart procedures in the GUI. Of course there have been adaptions and expansions, but about 80%-85% of the hard legwork has been done by Rex there. Only the PCO-tool is 100% DICE.

http://img84.imageshack.us/img84/8185/unled2pv.jpg

Issues to resolve: Terrain texture has an absurd tiling. Will have to see how to fix that. Rest was a lot of juggling with functions, but surprisingly easy. Rex has kept his scripts pretty tidy and well-documented (mostly) .
Post Reply