200 replies to this topic

[Amethyst Deceiver]

hmm.


i deleted all of the extra NVMI records in the NAVI and loaded my navmeshed test esp.

i did NOT finalize the tamriel cell that has my door teleporter, so the marker is just sitting on a red triangle in tamriels worldspace. i did finalize everything in the interior cell.



i cannot get the navmesh bug to happen at all.

[AntJam]

You can create a fresh interior cell by right clicking on an existing cell select edit to bring up the cell window, then right click on the cell list and select new. Creats a brand new empty pristine cell

[Amethyst Deceiver]

the same NVMI records show up on the file even when using a blank cell.

i noticed that a lot of the navmesh locations in the NVMI's match up to the initial warning errors when you first open the CK and skyrim.esm. almost all of those errors are some form of door. the one that concerns me the most is the helgen exterior ones, because one of the most common complaints in house mods is that helgen disappears after the mod is installed. this usually only happens when more than 1 house mod is installed, which leads me to believe these dirty NVMI records are overlapping and conflicting between mods

i know most people think these pop up errors are arbitrary and harmless, but i think the CK is writing the NVMI's because of these errors

whether or not this has anything to do with the navmesh bug is anyones guess, but all i know is that every time you hit the save button, your esp's navmesh info is getting polluted with a ton of crap.

Edited by Amethyst Deceiver, 15 April 2012 - 09:35 AM.

[ch0k3h0ld]

That is an interesting observation AD. When I look at the NAVI entries all I see is a bunch of hex. The only sense I have ever made out of it was to correlate a few of the entries to the NAVMs. I always wondered what all of those other entries were there for. How are you seeing the text entries?

[Amethyst Deceiver]

the first cluster of numbers are hexidecimal form id's

you can enter these form id's in text search in the ck, which will bring up the text

[Amethyst Deceiver]

for example in tessnip it will look something like

66 34 0D 00

which is the form id 000D3466

[ch0k3h0ld]

I knew those were the form ids but I never thought about entering them in the CK to see where they lead. I understand how you would search for objects, but how do you search for NAV entries?

[Amethyst Deceiver]

a navmesh IS an object

[SLuckyD]

I believe the NVMI subrecords, all contained in the single NAVI record, are essentially 'pointers' to the NAVMs... which are what contain the actual geometry, orientation, and merges. If you notice, each NVMI has the FormID of the NAVM it points to, the first 4 bytes in reverse the hex... 00123456 FormID would be 56 34 12 00 at the beginning of the NVMI. If you create a NAVM with a custom FormID, a new NVMI will be added to the NAVI record... as each NAVM always has one NVMI.

If you manually delete an NVMI subrecord, and if that cell is merged/linked to a navMeshed cell, it may regen that NVMI because it expects a placeholder there (linked by another NVMI I think). Kinda like it auto-gens entire copies of NAVM and NVMI of cells adjacent to one to finalize.. the plugin needs to have those pointers for some reason (implying that they are potentially altered in the process).

That would be why they are always there... and this is also why I believe our mods' interiors still had followers enter them - if the NAVM was deleted (for the buggy exterior), the bug is eliminated. BUT, if the corresponding NVMIs were NOT deleted (I know in my 1.524 version I didn't), then the game-engine is still pointing to the interior navMesh through that NVMI. I think it was getting the followers to come back out through that door was the problem... as now the NAVM being pointed to in the exterior has been deleted. If it works anyway, the NVMI may also contain door data independent of tri-data.

How multiple mods navMeshing the same area work is beyond my understanding.... they all have their own single NAVI record; each of which altering Vanilla's ONE record. Is the game-engine programmed to overlay multiple NAVI grids on top of each other? That's my best guess, as in testing mods which change the same Vanilla tri's... they both work uneventfully (NavMesh Bug notwithstanding), and Vanilla's keeps working as well. One theory I have regarding non-AI buggy behavior (such as disappearing statics) is the question of 'how many overlaid navMesh grids can the game-engine handle before errors occur?' This doesn't help NavMesh Bug (proper), as it still happens when NO competing mods are loaded... although perhaps simply the one mod competing with Vanilla is enough to trigger it in SOME cells.


Regarding my Riften tests: I did more with it and found yet even stranger happenings. Some instances of NavMesh could be fixed by the aforementioned 'merging into Vanilla' technique... BUT some of the 'cured' cells still have actors' AI broken. This doesn't happen all the time, but it does happen in the same cell each time it DOES (which is NOT the Anathema Cell of yore, stranger yet even).

Sometimes it's cured completely, sometimes one actor is affected, sometimes two or three... but their AI is perma-broken (not even scripting fixes it). Yet ONLY actors set to AI-patrol within THAT same cell only are affected. Sometimes deleting the broken actor and replacing it with fresh fixes it, sometimes merely making a clone of it fixes it... occasionally it can't be fixed at ALL... even after fire-bombing the entire plugin of anything related to that cell and rebuilding it all. This is VERY discouraging for something so time-consuming... it has no apparent reason as I do everything near-exactly the same each time. There HAS to be SOMETHING different.

I HAVE conclusively verified (to myself) that the navMesh geometry itself is NOT damaged by NavMesh Bug. When the Bug's triggered, it temporarily breaks the mergers between cell-borders... breaking any AI package which crosses it. I know the navMesh isn't damaged, because actors scripted back into their spots will perform their tasks perfectly... only actors whose AI package has them LEAVING a Bugged cell (my 'Anathema Cells') remains broken. Actors originating from outside the cell, will still travel INTO the Bugged cell.. but not come back out. I call this aspect of the NavMesh Bug "Roach-Motel Syndrome". [that command from Obliv really WOULD be helpful to prove this conclusively for all]

I've tried analysing the NAVM and NVMI... nothing striking stood out. I even compared a fixed NAVM to a broken one... identical (byte for byte). I looked at the ACTOR and IDLE MARKER refs (since replacing the actor fixes it sometimes)... I saw that their "Flags2:" is changed from the Vanilla 00000000 to 0000700 (with either an "E" or a "D" at the end). It seems that actors with broken AI have the 0s added back in... but changing them doesn't do a single thing... for ANY of them. I was just pointing out that SOME data is changed under certain circumstance... what else is changed, I have no idea (perhaps the NVMI or NAVM).

There was also some other seemingly non-important data which was changed: (when selecting the CELL record for the exterior) the XCLC subrecord has an "External Unknown" - Vanilla cells are all always the same (5504256), CK changes this number to something different each time (such as 5439744). Changing it back didn't visibly affect or fix anything.

Even if my potential fix above works to fix the cell-borders in general, the NavMesh Bug still affects actors not SPECIFICALLY scripted to return to a certain spot. This does NOT help mods such as Open Cities... it only seemed to work, but the actors' AI was still broken unless scripted otherwise. They would spawn in other spots... throughout the custom navMesh, but they wouldn't move around; each FTR finds them spawned on a different, random spot in the cell (ie: broken AI and wandering.. though the navMesh being fixed they don't always spawn in the same spot anymore).

I'm taking at least two things away from these findings - more anecdotal evidence that session-related functions may be causing parts of Family-NavMesh Bug (specifically relating to cell-border merges and AI packages), and the other thing being that it's becoming impossible to narrow down potential causes for it... as some aspects are seemingly random. I'm not sure I want to dedicate more time to the tedious testing of each little session-related difference... this is my FREE time, and it's quickly becoming something that shouldn't come free (not for nothin). In other words... I'm going back to modding to make good on some updates I've been talking about. (..much to SOME people's relief, I'm sure.. heheh)

One last comment... I HAVE always wondered if anyone has tried to FIX the Vanilla errors the CK shows each time it's loaded. Apparently, Ma Beth seems to think they're benign.. or they probably would have been fixed by now. Does anyone know more about this, or of a mod which purportedly fixes them? Also, I know I said I was gunna post video... but since the findings are spurious and unpredictable, I decided against going through all that drama.

[ch0k3h0ld]

Well, yeah. I guess I just never saw it as part of the objects listed in the Objects window, only as part of the respective cell it resides in. Sort of like saying that there is a PlayerHouseChest in the objects, but this particular PlayerHouseChest is only listed in my custom cell. I can search for it if I have my cell open. So what am I missing? It always seems that for every discovery I make there are a 1000 things I simply missed.

[Terra Nova]

@RealmEleven: Oblivion uses the path grid system. The only thing related to that which doesn't work properly, IIRC is auto-generate. You have to do all the pathing manually. At this point, I'd rather work with path grids.

sooo, i think this is too much of a coincidence.


ive been combing and picking apart the NAVI entries, and it seems that any time you create a new navmesh, it always has the same NVMI entries no matter what.

Helgen Exterior
Whiterun Exterior
Windhelm Exterior
Dawnstar... etc


Sound familiar?

How many times have you heard complaints from users saying Helgen/Whiterun/Dawnstar sanctuary etc. is completely missing or partially missing architecture?


i think it might be caused from duplicating AAAMarkers cell from the bethesda tutorial that i'm sure everyone did as well (since there is no way to "create" a fresh cell), which possibly causes a rather severe pollution on the NAVI entries by inserting so many unnecessary NVMI entries, REGARDLESS if you delete the initial navmesh and even if you delete the dirty entry in the CELL group.


and yes, this affects esm files as well (the divorced-bride-of-navmesh-bug-back-to-claim-alimony bug).



you want to know what's even more awesome than this?

every time you edit and save in the CK, the NAVI/NAVM records repopulate by themselves (they are always the same ones that keep coming back).


if this is really the cause of all the missing architecture, that means everyone has to clean their files in tessnip after saving in the CK... every *****ing time they make a change.




EDIT - i just did a quick test by duplicating a cell that has no existing navmesh XTestGrantInt5

oddly enough, it did not dirty edit the original cell like aaaMarkers does. i opened it in tessnip and sure enough there is no existing NAVI record.

i did a quick navmesh in the new cell, cover-edge and finalized. then checked the NAVI in tessnip again.

same NVMI entries.



why does the CK always write the same NVMI entries to those cells no matter what you do?

Serious question.. would this apply if you duplicate a different cell? I've always duplicated AbandonPrison cell, and deleted everything, including nav data(and making sure nothing is "left" in the cell..like this weird black squares)

Edit: hmm, think the answer in right in that post.

I think we are now getting somewhere.

Edited by Terra Nova, 15 April 2012 - 01:19 PM.

[ch0k3h0ld]

I'm just throwing this out there for the sake of discussion, but is it possible that there is an extra NAVI entry for each of the various WorldSpaces so the NAVM cell can connect to them? I would also suspect that those entries are part of the Vanilla mesh as well, but I don't know how to check. I've never tried loading Skyrim.esm into TESVSnip. If they are, they are probably not the sole cause of the problems but could still be connected. Unfortunately, at this point, it looks like anything could be connected. It is such as amazingly complex system.

I would also be very curious about fixing all of those errors in Skyrim.esm. I wish I had time. Maybe next weekend. (like I could fix them, probably just make things worse)

[SLuckyD]

For new interiors, I always r-click and make a new one, as AntJam said. I just tried it in the CK... and it does NOT generate ANYTHING except: the CELL grup, some subgrups leading to the custom CELL record, and an empty subgrup under that. No NAVI, no NAVM, no nothing whatsoever. I know that when you finalize a cell, it'll write NVMI subrecords for EVERY adjacent cells' navMeshes... is that what you're talking about?

If so, that's what I was referring to in the last post... about the NVMI being pointers and must always be present for cell-border merges to work. Now if the NVMI is what's damaged... and deleting it fixes the problem, what are the side-effects? I'm fairly certain the mergers to those NAVMs linked in the deleted NVMIs will revert to their Vanilla state... which would account for some forms of the NavMesh Bug (eg: spawning outside gates or other specific spots). This would mean your custom navMesh would essentially be ignored.. at least regarding cell-borders; it may be that the still-present NAVM (the actual geometry) functions properly (as ALL my tests show) but followers/actors set to exit that cell would likely balk at the idea.

I never compared the NVMI subrecords to one another (byte by byte)... a broken mod's versus the same Vanilla subrecord. I only compared NAVMs... as deleting them is what fixed the 1.524 CTDs. But if the CTDs were caused by somthing completely different than NavMesh Bug (which seems likely), the NVMIs may still be to blame. I left those NVMIs in my mod and it worked... but then again, my mod worked BEFORE that (no NavMesh Bug ever... only those 1.524 CTDs).

I'll take a look at this in a little while. What I said before about going back to modding was regarding my tedious experiments (which now contain over 40 different ESPs for Riften alone)... checking NVMI stuff doesn't really fall under that category.

[SLuckyD]

I finally found one... holy krap I just remembered why I didn't think twice about checking this earlier. I took like 10min just to FIND the matching NVMI... TESvSnip doesn't have a search option capable of doing what I need it to do I guess. But the subrecord I found points to NAVM 00026cd8 (in cell 43,-25... Tamriel-Riften). Here's what I've gleaned so far:

- Both NVMIs were the still the same size (Vanilla and the same NVMI in an ESP)... in this case they are 333 (hey, it was easy to scan for... unless you know of a better way to find something in a list of thousands?). Doing a byte-for-byte analysis, I found 12 blocks (2 consecutive bytes each) which were changed, and a single byte toward the end that was changed.

- That cell was NOT changed in the mod. It is a cell that is ADJACENT to a cell I altered (and finalized the navMesh for). The cells I altered do NOT merge into that cell (unless it did before I altered it), nor does it change anything whatsoever. It so happens that this cell is NOT considered a Bugged cell, though I'll double-check this in-game.

So why does that subrecord change if nothing's different in the cell? Even though the navMesh hasn't changed whatsoever, why does finalizing an adjacent cell change THIS cell's data? In case you were wondering, I copy-pasted the raw hex-data from TESvSnip into separate TXT files, then did a 'compare by content' on them (in Total Commander) - which finds and highlights any differences within a second... so at least for THIS I'm not relying on human pattern-recognition.

I'll keep looking for a cell which I DO change in the ESP... perhaps the blocks of altered data are different or in addition to those in the cell above; giving us a clue as to what goes wrong where. I'll also test that cell in-game - to see if it is, in fact, afflicted with our dramacita. I'll post back when/if I find anything more.

[Arthmoor]

Beth is too busy with a vision of people yelling FUS RO DAH at their TVs while we have the vision of an NPC just standing there with broken NavMesh unfortunatly.

LOL. It sure does seem that way sometimes, doesn't it?

They have enough people to do both. I'm hoping the lack of comments on their part just means that they're still working on it and have nothing new to say.

sooo, i think this is too much of a coincidence.


ive been combing and picking apart the NAVI entries, and it seems that any time you create a new navmesh, it always has the same NVMI entries no matter what.

Helgen Exterior
Whiterun Exterior
Windhelm Exterior
Dawnstar... etc


Sound familiar?

This would be expected actually. The NAVI record is global in scope and contains information on every navmesh in the game, including the ones you're adding to it. The game takes the NAVI data from every mod that has it and merges it into one record internally. If it didn't, then all it would take to spoil everyone's navmeshes everywhere would be one mod that changes one and is loaded last.

I think it would be pretty interesting though if you've found some correlation between how the NAVI record is assembled and merged with mod data and the resulting missing interiors in various places that have been modded. My alt-start mod does indeed add two fresh new cells to the game, both have navmeshes, and I've seen reports of the Helgen cave and the DB sanctuary outside Falkreath disappearing. It's left me to wonder if some of this is because people are refusing to include Update.esm as one of their masters. I always do, but plenty of other modders don't. An inconsistency in master data could definitely throw wrenches into things.

One thing you should not do though is start hacking data out of the NAVI record. It's intended to be there. Having it go missing will just end up breaking door links or something.

Regarding my Riften tests: I did more with it and found yet even stranger happenings. Some instances of NavMesh could be fixed by the aforementioned 'merging into Vanilla' technique... BUT some of the 'cured' cells still have actors' AI broken. This doesn't happen all the time, but it does happen in the same cell each time it DOES (which is NOT the Anathema Cell of yore, stranger yet even).

You're aware that the currently released copy of Open Cities already has vanilla merged navmeshes, right?

@RealmEleven: Oblivion uses the path grid system. The only thing related to that which doesn't work properly, IIRC is auto-generate. You have to do all the pathing manually. At this point, I'd rather work with path grids.

Even the auto-generate works so long as you don't attempt it on more than individual cells. Trying to do that over regions or entire worldspaces is asking for the CS to crash due to memory overload.

Also, yes, I'd be the happiest person on Earth if path grids came back and navmeshes were burned at the stake. I can path grid an entire city in under an hour and have it work the first time and never look back.

So why does that subrecord change if nothing's different in the cell? Even though the navMesh hasn't changed whatsoever, why does finalizing an adjacent cell change THIS cell's data?

It has to in order to update the edge connection data.

Edited by Arthmoor, 15 April 2012 - 03:43 PM.

[ajjones93]

Is there any news as to when the NavMesh bug will be fixed and released?
I'm just learning to mod now using the tutorials provided by Bethesda, but I plan on making mods that include a lot of quests and will be NPC heavy (which is no good as long as the NavMesh bug persists).

[Arthmoor]

No, not yet.

[SLuckyD]

Arthmoor: I was talking about my experiments in the Tamriel-Riften cells... done with just skyrim.esm - all that has nothing to do with OC except that I test some of the same cells. I only mentioned that it didn't help OC, like I had previously thought. I hadn't gotten the new version to see what it does in-game yet... I take it nothing that was changed helped?

About the NVMI having to be changed on an adjacent cell's finalize, even though that border hasn't changed - unless some kind of place-holder data which is session-specific is generated, I see no other reason for this to occur. I can understand having an identical copy of the subrecord in a mod (which sometimes is the case), but not modifying it when it doesn't have much to do with anything.

I found more NVMIs in Vanilla which correlate to one of my test ESPs... five were the same size in Vanilla as they are in the mod, some were changed and some weren't. Three of the cells had NVMI which was identical to the mod, and all three were adjacent to cells which were finalized. Of the two that had differences between custom and vanilla, one was in a cell that was changed and the other wasn't. The cell that was changed had 8 blocks of data changed (not the 12 blocks the unchanged cell had), and also had the single byte changed... but it was in the center of the data, not toward the end like the unchanged one.

All this means is... that I doubt this is gunna help us... especially how long it takes to do. It does prove that NVMI is both altered and left alone when adjacent cells finalize... which doesn't really help by itself, but at least we know. I'd be curious to know if those cells are buggy... I didn't place any actors in those cells. I'll test this and see if there's any difference between the ones changed (which would be buggy) and the ones that aren't (which I would expect to be normal).

[EDIT: I couldn't find any matching NVMI subrecords to NAVMs that I altered... I assume because their size changed. There were a total of five more records I couldn't find, but that could have been because I missed em (search doesn't work for this kind of thing, and I scanned for certain subrecord sizes). Either way, the cells/NVMIs I DID find were ones I have to go and place actors in to test.]

Edited by SLuckyD, 15 April 2012 - 07:18 PM.

[Arthmoor]

Arthmoor: I was talking about my experiments in the Tamriel-Riften cells... done with just skyrim.esm - all that has nothing to do with OC except that I test some of the same cells. I only mentioned that it didn't help OC, like I had previously thought. I hadn't gotten the new version to see what it does in-game yet... I take it nothing that was changed helped?

It made no difference. The copy that's out right now was actually an experiment I was doing on the hopes that altering a vanilla navmesh and extending it through the walls into the city proper would help. It didn't. Same exact set of issues. Balcony parties etc, NPCs stuck indoors unable to exit, followers not following. The whole thing was effectively a redo of the navmeshes for Riften in their entirety.

The 0.34 version doesn't have the follow-up I tried in Whiterun with the same methods so that's still composed of new navmeshes rather than extended vanilla ones.

You may well be right that it's the cell border data that's actually getting screwed up, but the bottom line here is that it's necessary data. You can't get those green bars any other way than to finalize a cell, and EVERY cell you work on has to be finalized or they become navmesh islands, which isn't very useful.

[Amethyst Deceiver]

btw, after cutting off my interior and worldspaces completely from tamriel (no navmesh link at all), and using only FT scripts to travel to and from the custom area, the followers do travel with you, as well as your horse (although i had to script their horse markers instead of the location based one)

however, dismissing followers inside the new custom area leaves them confused (standing around like the navmesh bug) and they don't know how to get back into skyrim (due to the inability to trace a path in the navmesh).

ah well, so much for that idea. it was worth a try though.

Edited by Amethyst Deceiver, 16 April 2012 - 03:42 AM.

[Rohugh]

Post limit.