Difference between revisions of "Data File"

3,255 bytes added ,  11:11, 6 January 2020
imported>Qazaaq
imported>Braydoge
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''File Extension:''' <code>.esm</code>, <code>.esl</code>, <code>.esp</code>, <code>.fos</code>, <code>.fid</code>, <code>.fud</code>, <code>.fvd</code>, <code>.ccc</code>
'''File Extension:''' <code>.esm</code>, <code>.esl</code>, <code>.esp</code>, <code>.fos</code>, <code>.fid</code>, <code>.fud</code>, <code>.fvd</code>, <code>.tes</code>, <code>.ccc</code>


A '''Data File''' acts as a database of all of the data for the world, including object data, dialogue, gameplay settings, object placements, AI settings, landscape, [[Cell]]s, etc.
A '''Data File''' acts as a database of all of the data for the world, including object data, dialogue, gameplay settings, object placements, AI settings, landscape, [[Cell]]s, etc.
Line 9: Line 9:
<code>.esm</code>
<code>.esm</code>
<BR>
<BR>
The [[Creation Kit]] will not create master files without additional enabling the Creation Kit's built in [[Version Control]] system.
The [[Creation Kit]] will not create master files without enabling the Creation Kit's built in [[Version Control]] system.
The merging process takes the interim ESP data and merges into a master ESM file to be used by the collaborators.
The version control merging process takes the interim ESP data and merges it into a shared master ESM file.
The shared ESM is used by collaborators to create their own interim ESP data.
Some community made tools exist which are capable of merging ESP files to masters in a similar manner as Creation Kit version control system as well.


==== Removing ESM Files ====
Usually, most authors who want to create ESM files will create an ESP file for editing in the Creation Kit.
While ESP files can usually be removed from gameplay simply by de-activating them, ESM files cannot be removed in this way.
Once the ESP file is ready for public release, they will manually convert the file meta and extension to an ESM master.
In short the [[Creation Engine]] refuses to remove an ESM from the save game.
The ESP should be excluded from public releases when the intention is to release a master file.
However, this can easily overcome by modifying the savegame to think that the ESM was actually an ESP -- at which point the game engine will happily remove it.
(See Wrye Bash: Disabling Masters.)


The ESP may be kept by the author indefinitely, or manually converted to and from ESP as needed.


== Light Master ==
== Light Master ==
<code>.esl</code>
<code>.esl</code>
<BR>
<BR>
[[Fallout 4|Fallout 4 v1.10+]]
The ''Light Master'' (.esl) file is a new optimized file format introduced in the [[Fallout 4]] v1.10 update.
<BR>
The ''Light Master'' (.esl) file is a new optimized file format introduced in the v1.10 update.
Using ''Light Master'' files will allow more plugins to be loaded than the previously possible due to the 8-bit plugin ID limit of '''255'''.
Using ''Light Master'' files will allow more plugins to be loaded than the previously possible due to the 8-bit plugin ID limit of '''255'''.
A ''Light Master'' does this by using more of the ''Form ID'' range for load order, while also sacrificing it's maximum capacity for form records.
A ''Light Master'' does this by using more of the ''Form ID'' range for load order, while also sacrificing it's maximum capacity for form records.
Line 32: Line 31:


==== Light Master Notes ====
==== Light Master Notes ====
*''Light Master'' files released by Bethesda Game Studios ([[Creation Club]] releases) are flagged as master files and always load immediately after the official Bethesda masters, in the order specified by the game's <code>.ccc</code> file. These .esl files cannot be deactivated.
* ''Light Master'' files released by Bethesda Game Studios ([[Creation Club]] releases) are flagged as master files and always load immediately after the official Bethesda masters, in the order specified by the game's <code>.ccc</code> file. These .esl files cannot be deactivated.
*''Light Master'' files released by modders are flagged the same as ESP files, but these '''also''' load with the master files, in the order specified in <code>plugins.txt</code>. These .esl files will need to be activated before the game will load them.
* ''Light Master'' files released by modders are flagged the same as ESP files, but these '''also''' load with the master files, in the order specified in <code>plugins.txt</code>. These .esl files will need to be activated before the game will load them.
*''Light Master'' files created by modders cannot use standard ESP files as masters or this will cause the ESP file to also load before the .esl in the master files section. This is new behavior for the game engine as of Skyrim Special Edition v1.5.3.
* ''Light Master'' files created by modders cannot use standard ESP files as masters or this will cause the ESP file to also load before the .esl in the master files section. This is new behavior for the game engine as of Skyrim Special Edition v1.5.3.
** Any dependencies of the normal plugin a light master depends on are '''not''' loaded early.
** Any dependencies of the normal plugin a light master depends on are '''not''' loaded early.
** Creating light masters that are dependent on normal plugins is highly discouraged.
** Creating light masters that are dependent on normal plugins is highly discouraged.


==== Light Master Budgets ====
==== Light Master Budgets ====
A ''Light Master'' file has a budget of no more than 2,048 forms, with Form IDs <code>0x800</code> to <code>0xFFF</code> available for use.
A ''Light Master'' file has a budget of no more than 4096 forms (prior to version 1.34: 2048 forms), with Form IDs <code>0x000</code> to <code>0xFFF</code> available for use.
A absolute maximum of 4096 ESL files can be loaded by the engine at once, but in practice this is difficult to achieve.
A absolute maximum of 4096 ESL files can be loaded by the engine at once, but in practice this is difficult to achieve.
331 plugins can be loaded that each contain 2048 [[ALCH]] records, but 970+ plugins can be loaded that each contain one [[CELL]] record and 2047 [[REFR]] records.
331 plugins can be loaded that each contain 2048 [[ALCH]] records, but 970+ plugins can be loaded that each contain one [[CELL]] record and 2047 [[REFR]] records.
Line 67: Line 66:
Be sure to save ESP files after running ''Compact Active File'', because otherwise compacting again with new forms might give different Form IDs for forms already in a game save.
Be sure to save ESP files after running ''Compact Active File'', because otherwise compacting again with new forms might give different Form IDs for forms already in a game save.
As long as development is done on compacted ESP files that match released ESL files, forms can be added under the 2,048 form limit, and existing forms can be edited without issue.
As long as development is done on compacted ESP files that match released ESL files, forms can be added under the 2,048 form limit, and existing forms can be edited without issue.
==== Light Master to Plugin Conversion ====
While not directly supported, an ESL can be converted back into an ESP for direct editing.
As of '''Update v1.10+''', simply rename the ESL file extension to <code>*.esp</code>.
With the renamed file extension, the [[Creation Kit]] will load the ESL as a standard ESP plugin.
When ''Active Plugin'' is saved, the [[Creation Kit]] will complete the conversion by removing the master header flag.
The ESl is now converted into an ESP plugin file.
The ''Convert Active File to Light Master'' command may be used to publish the file as an ESL light master again.
==== Light Master to Plugin Manual Conversion ====
Before '''Update v1.10''', manually converting a light master to a plugin was not supported by the [[Creation Kit]].
To manually convert a light master to a plugin, do the following.
# Rename the file back to <code>.esp</code> and open it for editing with [[xEdit]],
# In the '''Header''' section, edit the flags to trigger a resave. Simply choosing the '''ESM''' flag on and off again will suffice.
# Save the modified plugin. It will then be a binary identical to the original <code>*.esp</code> it was created from.


== Plugin ==
== Plugin ==
Line 91: Line 72:
A plugin ESP file created in the [[Creation Kit]] may only rely on data from itself and any master ESM files.
A plugin ESP file created in the [[Creation Kit]] may only rely on data from itself and any master ESM files.
It's also possible to create plugins that depend on other plugins using third-party tools.
It's also possible to create plugins that depend on other plugins using third-party tools.
*Plugins are '''not''' required to contain any data.
* Plugins are '''not''' required to contain any data.
*Plugins are '''not''' required to have a master if they are empty or add '''new''' content.
* Plugins are '''not''' required to have a master if they are empty or add '''new''' content.
*Plugins can only refer directly to data from a master file, though they can refer to multiple master files.
* Plugins can only refer directly to data from a master file, though they can refer to multiple master files.
*Plugins can only refer to information in another plugin indirectly by using scripts.
* Plugins can only refer to information in another plugin indirectly by using scripts.


=== Light Plugin ===
Also known as "ESL-Flagged ESP" or "ESPFE", this is a combination of a light master and a plugin. It's a file with the <code>.esp</code> file extension, but the '''ESL''' header flag.
* Like light masters, light plugins have a budget of at most 4096 forms, between <code>0x000</code> and <code>0xFFF</code>, and are merged at runtime into the <code>0xFE</code> slot.
* Light plugins are loaded together with regular plugins. This means, they can both override other plugins, and be made to require plugins as masters without negatively affecting the load order.
* In certain circumstances, regular plugins can be made light without breaking compatibility to existing saves.
A light plugin can be created using various means:
* Use the [[Creation Kit]] to create a light master as described above. Then, change the resulting ESL file's extension to <code>.esp</code>
* Load the plugin in question in [[xEdit]]
** Once loaded, right-click the file, select "Compact FormIDs for ESL".
** In the '''Header''' section, add the '''ESL''' flag.
** Save the file.
* [[Vortex]] is capable of making plugins "light" from it's Plugins tab.


== Game Save ==
== Game Save ==
Line 102: Line 97:
A Fallout Save (.fos) is used to restore the games state between sessions.
A Fallout Save (.fos) is used to restore the games state between sessions.
The [[Creation Kit]] cannot create, edit, or view save files.
The [[Creation Kit]] cannot create, edit, or view save files.
Uninstallation of any data file from a game save is not recommend, and officially '''not supported'''.


==== Removing Master Files ====
While ESP files can '''sometimes''' be uninstalled from game play without consequence, an ESM file can never be removed in this way.
In short the [[Creation Engine]] refuses to remove an ESM from a ''game save''.
However, this can be overcome by modifying the ''game save'' meta to think the missing ESM was actually an ESP.
Once the master list has been edited on the ''game save'' file header, the game engine will happily remove it.
[[Wrye Bash]] provides a ''game save'' editor capable of disabling masters.


== Version Control ==
== Version Control ==
Line 110: Line 113:
These files are used by the [[Creation Kit]] to track and merge changes between file versions.
These files are used by the [[Creation Kit]] to track and merge changes between file versions.


== Modifying Other Mods ==
While the Creation Kit allows ESPs to use and modify ESMs, it will not allow similar actions for ESPs.
The basic problem here is that the Creation Kit refuses to allow an ESP to become a master of an ESP.
And without that master relationship, it's impossible for the new ESP to refer to anything in the old ESP (and thus is unable to use or modify it).
However, it's possible to trick the Creation Kit into remembering another ESP as a master.
Once this is done, then the new mod is able to use and modify the old ESP (almost) as if it were a regular ESM file.
See De-Isolation Tutorial link at the bottom of this page.
Also of note is that ESMs can modify other ESMs -- within certain limits.
The main limit seems to be that one ESM cannot add items to worldspaces defined by another ESM.
Or rather, it can't do so safely -- since doing so causes landscape and buildings to disappear during gameplay.
== Data File Conversions ==
The [[Creation Kit]] does not provide a straight forward conversion between data files, though it does have some limited methods for conversion.
Several community made tools also exist which allow easy conversion between ESM, ESL, and ESP files.
==== Light Master to Plugin Conversion ====
While not directly supported, an ESL can be converted back into an ESP for direct editing.
As of '''Update v1.10+''', simply rename the ESL file extension to <code>*.esp</code>.
With the renamed file extension, the [[Creation Kit]] will load the ESL as a standard ESP plugin.
When ''Active Plugin'' is saved, the [[Creation Kit]] will complete the conversion by removing the master header flag.
The ESL is now converted into an ESP plugin file.
The ''Convert Active File to Light Master'' command may be used to publish the file as an ESL light master again.
==== Light Master to Plugin Manual Conversion ====
Before '''Update v1.10''', manually converting a light master to a plugin was not supported by the [[Creation Kit]].
To manually convert a light master to a plugin, do the following.
# Rename the file back to <code>.esp</code> and open it for editing with [[xEdit]],
# In the '''Header''' section, edit the flags to trigger a re-save. Simply choosing the '''ESM''' flag on and off again will suffice.
# Save the modified plugin. It will then be a binary identical to the original <code>*.esp</code> it was created from.


= Tooling =
= Tooling =
*The [[Creation Kit]] is the primary tooling for working with data files.
* The [[Creation Kit]] is the primary tooling for working with data files.
*[[xEdit]]
* [[xEdit]]
*[[Wrye Bash]]
* [[Wrye Bash]]
*There are other community software projects that provided tooling for working with data files.
* There are other community software projects that provided tooling for working with data files.
 


= Notes =
= Notes =
*By tradition, the file extensions stand for '''E'''lder '''S'''crolls '''M'''aster and '''E'''lder '''S'''crolls '''P'''lugin.
* By tradition, the file extensions stand for '''E'''lder '''S'''crolls '''M'''aster and '''E'''lder '''S'''crolls '''P'''lugin.
 


= See Also =
= See Also =
*[[Version Control]]
* [[Version Control]]
*[[Game Files]]
* [[Game Files]]


== Links ==
== Links ==
Anonymous user