216 replies to this topic

[Leviathan1753]

SkyProc

Java-based Skyrim Processing Library

What is SkyProc?

SkyProc is a Java library that offers easy to use Java API for importing, manipulating, and exporting mods.

The SkyProc library offers Java programmers the power to create and edit objects that represent Skyrim mods and records. It is able to import mods, or even an entire load order, and give easy access to the records inside. Programmers can then make any changes they wish, and export a working Skyrim patch that is customized to every user's load order.

Its purpose is to facilitate third party creation of smart programs that create custom patches based on any given load order. For many mods, this will help reduce, or completely eliminate conflicts. It is currently in early development stages, offering limited access to several GRUPs.


What can SkyProc be used for?

It can be used to make extremely smart and versatile patchers capable of adapting to any load order. It can be used for things such as:

• Customization Patchers
  Realistic Lighting is an example of a mod that uses SkyProc for customization capabilities. Instead of making several esps with varying hardcoded settings, the RL SkyProc patcher reads in an INI file, and adjusts the Realistic Lighting.esp patch to contain the custom settings defined by the user. This gives RL an unprecedented amount of customization for users.
  
  .
• Addition Patchers
  Automatic Spells is an example of a mod that uses SkyProc to seamlessly add things to the world, without causing incompatibility issues. It reads in the NPC_ and SPEL groups from all the active plugins in a user's entire load order, it then loops over each, adding spells to NPCs based on the spells difficulty, and the NPC's spell skill stats.
  
  A comparable patch made by hand would take an extremely long time to make. In addition, that hard-coded patch would only ever affect Skyrim.esm. If a user wanted the same thing done to say, Midas, the modder would have to do a customized patch for Midas by hand as well. If Midas ever came out with a new set of spells, the modder again would have to adjust by hand.
  
  Doing this type of mod using a SkyProc patcher will work on any mod fed to it, no matter if it's Skyrim.esm, Midas 1.0, Midas 2.0. It will still read in the GRUPs, do its calculations, and hand out the spells accordingly. What this results in is a single patcher that works for all mods. If someone has only Skyrim.esm, it will work. If someone has Midas as well, it will not only will work with Midas, but it will add Midas' spells to Skyrim.esm NPCs, as well as NPCs from other mods.
  
  .
• Preparation Patchers
  Dynamic Leveled Lists is an example of a mod that uses SkyProc (in this case, an ancestral embedded version) to prepare the user's mod setup for in-game scripts. DLL is a spawn system overhaul that needs all the Leveled Lists to be organized in a special way, and have DLL-specific dummy entries with scripts attached to them. DLL comes in two parts. The first is a SkyProc-style patcher that processed the user's load order and creates a patch with all the Leveled Lists in proper format. The second part is a set of in-game scripts that take advantage of the preparation and offer much more dynamic spawning probabilities.

There are probably countless other types of applications that can be made with SkyProc, as it loads in mods, and then lays them on a platter for you to do whatever you want with them.

The possibilities are really up to you as the programmer to flex your coding muscles and take advantage of the power SkyProc gives you.




For more information, documentation, and help tutorials, please visit:

SkyProc wiki

Edited by Leviathan1753, 17 June 2012 - 01:22 AM.

[Leviathan1753]

Sorry for the extreme neglect, faithful SkyProc users! The forums firmly decided that I didn't need to receive any emails about posts being made on it. 8P I've been completely oblivious to the posts over the past two weeks.

In addition, the official distribution hasn't been updated in quite some time, while the open source repository is constantly being updated for specific user requests. I'm sure this is the source of a lot of discrepancies in views of what SkyProc was capable of or not. I'll be updating the official distribution and javadocs soon to get everyone on the same page.

Here's my attempt at combing through all the posts and trying to give a good answer:

Code as I last saw it:

ModListing Perked = new ModListing("NPCPerked", false);

FormID PerkListID = new FormID("00196E", Perked);

FLST PerkList = (FLST) SPGlobal.getGlobalPatch().getFormLists().get(PerkListID);
for (FormID p : PerkList.getFromEntries()) {
   for (NPC_ n : patch.getNPCs()) {
	  n.addPerk(p, 1);
   }
}

I'm assuming you have a mod called NPCPerked.esp that is being imported.

The cause of your issues is the FLST PerkList = (FLST) SPGlobal.getGlobalPatch().getFormLists().get(PerkListID); line
This is because you are telling SkyProc, "Get me the global patch (the patch to be exported), and get me the FormList with FormID PerkListID."

Well.. the export patch starts out empty, so it doesn't have the FormList you're looking for, so it returns null.. causing the error on the next line.

What you probably want is

FLST PerkList = (FLST) SPGlobal.getDB().getMod(Perked).getFormLists().get(PerkListID);

This will get you the NPCPerked.esp mod as you were probably intending.

There is also one more thing you need to handle. You've added the perk to the NPC from Skyrim.esm... however the export patch is still empty. You'll need to add the modified/desired NPCs to the export patch.

ModListing Perked = new ModListing("NPCPerked", false);

FormID PerkListID = new FormID("00196E", Perked);

FLST PerkList = (FLST) SPGlobal.getDB().getMod(Perked).getFormLists().get(PerkListID);
for (FormID p : PerkList.getFromEntries()) {
   for (NPC_ n : patch.getNPCs()) {
	  n.addPerk(p, 1);
      SPGlobal.getGlobalPatch().addRecord(n);
   }
}

If you add the same NPC twice it'll still all work peachy. You just want to make sure each modified NPC gets added at least ONCE.

Please let me know if anything else doesn't make sense. And feel free to poke me, message me, scream at me, or even beg Pluto for my Gmail name to message me on gTalk if I'm not responding. 8P

Edited by Leviathan1753, 17 June 2012 - 03:40 AM.

[Leviathan1753]

SkyProc 1.4 Beta Release

• Added a new consistency system that keeps track of old EDID <-> FormID pairs. This makes it safe to erase generated .esp files without risking a ruined savegame.
• Added SaveFile functionality
• Added a new GUI setup called SUMGUI (featured in Automatic Variants). SkyProc coders can now easily set up their own GUI with their own custom settings. The GUI setup will automatically handle saving/loading settings.
• Added the beginnings of a SUM (SkyProc Unified Manager) management interface/program. This will eventually serve as a coordination program between SkyProc patchers and streamline the patching process for end users who run multiple skyproc mods.
• Added a Nifty class that offers lots of "nifty" functions that streamline common or tedious jobs for you (such attaching a script to a race as a racial spell).
• Added a lot more functionality, support, and accessors for many record types. Check the Javadocs for detailed listing of functions offered now.
• Added/Tweaked a lot of LevGUI classes

In short:
Lots of new functionality, bugfixes, and supported record types. Too many to list. Jump in, check the javadocs, and always feel free to ask questions or send me messages. I'm here to help and make things happen. 8)

[AliTheLord]

Wohoo, cheers .

[Arkangel]

So, I've been contemplating making my own SkyProc patcher for crafting recipes, basically something akin to Lorecraft, but in the form of a dynamic SkyProc patch. Haven't quite worked out the details yet, but it'd basically make all craftables require you to find a corresponding recipe before you could craft them. These recipes could then be found all over Skyrim. I'm not sure how many people would use this, but I'd like to do it as a pet project anyway.
However, when I fired up the startet project from the SkyProc google docs page, I realized that GRUP_TYPE doesn't contain COBJ. Am I correct in assuming that this means craftables records aren't yet supported?

[Lilliputian]

...it'd basically make all craftables require you to find a corresponding recipe before you could craft them. These recipes could then be found all over Skyrim. I'm not sure how many people would use this, but...

Oh, oh, I passed on Lorecraft exactly because it didn't include this functionality you described. You have at least one person who would absolutely be interested!

I suppose this kind of post could be done in PM, but it also shows that there is interest in SkyProc's feature set, which is probably good for the tool and it's author.

[charon711]

So, I've been contemplating making my own SkyProc patcher for crafting recipes, basically something akin to Lorecraft, but in the form of a dynamic SkyProc patch. Haven't quite worked out the details yet, but it'd basically make all craftables require you to find a corresponding recipe before you could craft them. These recipes could then be found all over Skyrim. I'm not sure how many people would use this, but I'd like to do it as a pet project anyway.
However, when I fired up the startet project from the SkyProc google docs page, I realized that GRUP_TYPE doesn't contain COBJ. Am I correct in assuming that this means craftables records aren't yet supported?

I would download that in a heartbeat! It made no sense that just because i have a perk i magically know how to construct all the weapons and armors associated with it. Either have to find the recipe or have some one teach you it! maybe as a reward in some quests your taught how to craft some things... like the smithing tutorial or finding that book for the apprentice black smith in that one town. forget which town it was but the head smith was a female orc and the apprentice was a imperial. She could teach you how to craft a random orcish item. Stuff like that would be pretty cool!

[Arkangel]

It made no sense that just because i have a perk i magically know how to construct all the weapons and armors associated with it. Either have to find the recipe or have some one teach you it! maybe as a reward in some quests your taught how to craft some things... like the smithing tutorial or finding that book for the apprentice black smith in that one town. forget which town it was but the head smith was a female orc and the apprentice was a imperial. She could teach you how to craft a random orcish item. Stuff like that would be pretty cool!

I suggest you look into Lorecraft, it already does something similar for vanilla recipes (and offers lots of patches for other crafting mods). I think it adds it's recipes to the world via books that can be found in loot, don't think you get anything by completing quests though. This is also probably not something that'd be in the scope of my planned patcher, as it'd just look through all craftables in your mod list and dynamically add recipes throughout Skyrim, not add specific recipes tied to quests, although I might consider doing something special for vanilla features if I ever get a stable base patcher going.

[Plutoman]

COBJ's are not yet supported, but could be added. How much experience do you have in programming? What you could do is sort through items and add recipes for each item. You could probably create a base recipe and duplicate it inside SkyProc, or I could see about a constructor.

Send me a PM and I can see about adding it in.

[Uky]

I'm having a bit of difficulty with FormIDs. I'm working on a patcher that processes ENCH records. When I call the getBaseEnchantment() method on an ENCH record the FormID returned doesn't seem to have a master. If I try to use the returned FormID to retrieve the base enchantment record, it fails.

Example code:

  // this works
  FormID baseFid = new FormID("10FB70Skyrim.esm");  // EnchArmorFortifyAlchemyBase
  System.out.println(baseFid);

  MajorRecord base1 = SPDatabase.getMajor(baseFid, GRUP_TYPE.ENCH);
  System.out.println(base1);


  // this doesn't work...
  FormID enchFid = new FormID("08B65BSkyrim.esm");  // EnchArmorFortifyAlchemy01
  System.out.println(enchFid);

  ENCH ench = (ENCH) SPDatabase.getMajor(enchFid, GRUP_TYPE.ENCH);
  System.out.println(ench);

  FormID baseFid2 = ench.getBaseEnchantment();
  System.out.println(baseFid2);

  MajorRecord base2 = SPDatabase.getMajor(baseFid2, GRUP_TYPE.ENCH);
  System.out.println(base2);

Example console output:

10FB70Skyrim.esm
[ENCH | 10FB70Skyrim.esm | EnchArmorFortifyAlchemyBase]

08B65BSkyrim.esm
[ENCH | 08B65BSkyrim.esm | EnchArmorFortifyAlchemy01]
0010FB70
null

Am I doing something wrong or is this a bug?

edit: Oh, and I'm using the latest version of SkyProc from the Git repo:

• Revision: e521a4055d20 "Fixed an NPC_ set(NPCStat) bug" (Jun 27)

Edited by Uky, 01 July 2012 - 11:38 AM.

[Grimmykins]

Thanks for your work, Lev!

[Antagonistic]

Hmm, what about meshes?

I havn't looked at them in detail, but is it even theoretically possible for CBBE and UNP to use SkyProc to automatically modify clothes to use their bodies? This assumes that clothing meshes can be adapted via moving around vertices and bones, and any additional bones that need to be created can be computed. This will NOT even be possible if the effort to adapt vanilla->custom bodies require operations that cannot simply be computed, which is very possible.

[Plutoman]

I'm having a bit of difficulty with FormIDs. I'm working on a patcher that processes ENCH records. When I call the getBaseEnchantment() method on an ENCH record the FormID returned doesn't seem to have a master. If I try to use the returned FormID to retrieve the base enchantment record, it fails.

Am I doing something wrong or is this a bug?

edit: Oh, and I'm using the latest version of SkyProc from the Git repo:

• Revision: e521a4055d20 "Fixed an NPC_ set(NPCStat) bug" (Jun 27)

I'll take a look at it. Seems odd to have that.. The base enchantment should have been imported.

Antagonistic, I doubt that's possible. I have a feeling it takes more than that, and that's delving into a lot of nif file management which is only supported for textures. It'd have to be totally decoded and written up - something beyond my skills. The existing API would be of very little use.

Edited by Plutoman, 04 July 2012 - 12:27 PM.

[Leviathan1753]

@Uky
Thanks! That was definitely an issue caused by a bug. 8(
I'll get a SkyProc patch up very soon to address it.

Edited by Leviathan1753, 09 July 2012 - 01:30 PM.

[Leviathan1753]

SkyProc 1.4.1 Release

• FormID standardization bugfixes
• GLOB support

Thanks Uky for alerting me to the issue. 8)

[ishmaeltheforsaken]

If I wanted to make duplicates of forms and then manipulate those duplicates and save it into a new .esp, how would I do that?

[Leviathan1753]

Heya! It wouldn't be too bad.

Let's say you wanted to duplicate ALL armors in the game and set their pickup sounds to be None. (why? who cares) It would look something like this:

for (ARMO armor : merger.getArmors()) {
   ARMO armorDup = (ARMO) patch.makeCopy(armor, armor.getEDID() + "_DUP");
   armorDup.setPickupSound(FormID.NULL);
}

The program will then:

• Import all the armor records,
• Iterate through all of them
• Make a duplicate that originates from your output patch
• Set the duplicates pickup sound to None
• Export the patch

Done!

This code is assuming you use the new skyproc starter project which handles importing/exporting code for you...
I am updating SkyProc to version 1.5 and added a newly updated Starter project. 8)
It was all due for an update, anyway.

In this new SkyProc Starter Project,
you can literally paste that code in the runChangesToPatch() function, set the importRequests() function to be ARMO records.. and the patcher is complete and usable! I tried to make it as out of the box as possible. 8)

I know the documentation on the wiki is terribly old and outdated. 8( Most of the SkyProc authors just talk with me on a personal basis to Q/A. Feel free to shoot me a PM if you want to talk more SkyProc stuff in detail; Or here works!

Edited by Leviathan1753, 06 September 2012 - 02:11 AM.

[ishmaeltheforsaken]

Updating is exciting! When do you think that will be?

[Leviathan1753]

Right now! Was just tweaking the documentation. 8)
SkyProc 1.5 Released
- Added a new starter project that conforms to SUM (SkyProc Unified Manager; Unreleased manager program) standards.
- Reworked how condition records and embedded scripts were implemented. Thanks to Arkangel for hand-parsing the embedded scripts!
- Patch now adds as masters any other mod that modified any record it contains. (symbolizing that those mods had a part in its patch)
- Updated SUM and SUMGUI code, GUI, and protocols.
- Now reads STRINGS files from BSAs (Dawnguard)
- BSA parsing more robust; Handles bad file type flags.
- Removed masks
- Bug fixes and internal protocol adjustments



If you open and run the starter project, it opens the GUI and displays. Patching makes an empty patch.

All you have to do is tell the program what to import in importRequests(),
and then put in your custom code in runChangesToPatch()... which is the hard/creative part. 8)

The rest is handled by the starter project and SkyProc.. minus any customization you want to throw into the GUI make it unique to your project. 8)


Again, let me know if you have any questions. 8) I'm always up for talking/helping/discussing/troubleshooting.
Also, SkyProc doesn't support ALL records, or give access to everything in the records it does support. So if somethings missing that you want, just shoot me a message and I'll get it to you ASAP.

Edited by Leviathan1753, 06 September 2012 - 02:15 AM.

[VadersApp]

I want to get into skyproc. But the wiki seems to be outdated. It would be cool if you can tell how to setup up a working project in netbeans. In the wiki there are 2 libraries that aren´t in the distribution. I would be happy if you can give some help. I would like to port functions of my code to skyproc. It will make working on RBS much easier i think.

Edit:Nevermind, got it running. Needed newer JDK not the one coming with netbeans

Edit2: Changed Project Source to JDK7, compiled the raw starter project, started the patcher, after clicking on patch. the patcher closes. Shouldn´t be there a message like patch completed? I think the patcher crashes, before building the patch.

Edit3: sorry, have given up for now. If you want new people start using skyproc you should update the wiki to the latest version of skyproc. Also please be so kind and add sample code that shows how to read in values and write it to the patch file. For example change the races for every human npc. That is what i wanted to do in a first step

Edited by VadersApp, 16 September 2012 - 08:51 AM.

[Chaky]

Skyproc noob enthusiast here.

I've noticed that:

1. all of the tweaked records don't carry over "compressed" record flag bit. Is it doable?
2. with NPC_ records, there seems to be an issue with SHRT (Short Name) strings. They either turn to be jibberish, or not strings at all.

Btw, I'm using skyproc 1.6 on 64-bit JDK 1.7 + NetbeansIDE 7.2.1

P.S. Talk to me like I'm 5-year old. That's my level of understanding java. (but I'm growing fast )

Edited by Chaky, 15 November 2012 - 02:10 PM.

[Plutoman]

Were the short names modified to be encoded in the strings files? I wonder if that caused it. Or the reverse, if they aren't in the strings files.. Sounds like something there, though.

[Chesko]

With Last Seed, I am trying to solve some difficult problems and I'm wondering if SkyProc would be the best tool for the job.

The primary issue is that I would like to replace hundreds and hundreds of objectreferences throughout the world with objects that I have created on a semi-random basis (semi-random meaning, replacing objectrefs based on a set of parameters, with some degree of variance). For instance, I would like to replace 85% of the apples found in cells with the Location keyword of LocTypeFort with a new apple of my creation. Is this something SkyProc can be made to do?

Thanks

[AliTheLord]

With Last Seed, I am trying to solve some difficult problems and I'm wondering if SkyProc would be the best tool for the job.

The primary issue is that I would like to replace hundreds and hundreds of objectreferences throughout the world with objects that I have created on a semi-random basis (semi-random meaning, replacing objectrefs based on a set of parameters, with some degree of variance). For instance, I would like to replace 85% of the apples found in cells with the Location keyword of LocTypeFort with a new apple of my creation. Is this something SkyProc can be made to do?

Thanks

Yes, but CELL records haven't been added to SkyProc yet. If you ask Lev nicely he may get around to adding CELL records . Once they're added it will open up a world of probabilities anyway .

[Chesko]

Cool, thanks.

[zilav]

It can be easily done wth a script in TES5Edit. PM me details if you want.

[charon711]

Rather than make a thread I thought it would be better to place a question/request here. Would it be possible to make a SkyProc patcher that could take the script from a mod like Dual Sheath plus Shields on Back (DSpSoB) and place it on all appropriate weapons/shields from other mods? It's killing me to not have Immersive Weapons work with this and it would take me forever to patch it myself (if I can figure that out, I am a noob with the CK). That way it would be quick, easy and accessible to others that would want to patch other mods.

Not sure if thats even possible but if it is could some on PLEASE FOR THE LOVE OF ALL THAT IS GOOD AND DECENT! PLEASE MAKE A PATCHER THAT WOULD DO THAT! (end dramatic all caps )

EDIT

*SIGH* I think what I want would be impossible, I didnt realize that there is a lot that has to be done in NifScope to get this to work.

Edited by charon711, 09 December 2012 - 01:59 AM.

[BarbasTheDog]

Maybe it's a silly question to ask, but can SkyProc mess up with Default Objects? I think it's DOBJ or something. This would be most valuable for... Lots of stuff! (Assuming it's even possible) If you could make it possible, me and a LOT of other modders would love you for ever!

[ArcaneSmith]

So, I've been contemplating making my own SkyProc patcher for crafting recipes, basically something akin to Lorecraft, but in the form of a dynamic SkyProc patch. Haven't quite worked out the details yet, but it'd basically make all craftables require you to find a corresponding recipe before you could craft them. These recipes could then be found all over Skyrim. I'm not sure how many people would use this, but I'd like to do it as a pet project anyway.
However, when I fired up the startet project from the SkyProc google docs page, I realized that GRUP_TYPE doesn't contain COBJ. Am I correct in assuming that this means craftables records aren't yet supported?

@Arkangel

So, I'm the author of Lorecraft and i'm REALLY interested in making (well i'll have to learn Java!! oO) or getting someone to help me and make a patcher to convert item plugins. I've PMed you.


@Leviathan

Hey there! So, is it possible to make something that does at least the following?:

(Assuming the esp being patched already has COBJs)

- Create a different PERK for each COBJ in the ESP being converted
- Add the corresponding PERKs to the COBJs as a HasPerk condition aimed at the Player (oO)
- Copy a template BOOK from the Lorecraft.esm into the esp being patched
- Make one BOOK for each PERK, based on this copied template
- Add each PERK to a property in the attached script on each corresponding BOOK (oO)
- Fill a formlist with the created BOOKs
- Copy a Quest from Lorecraft.esm into the esp being patched
- Add the created formlist to a property in the quest script (oO)


So yeah, I doubt you can change script properties, but I hope the rest is doable =)

Thanks!

[Plutoman]

Script properties are editable, anything stored in properties is editable.. the actual script isn't, but the properties are. You might've had a reply from him already though.