Difference between revisions of "Data File"

From the Fallout4 CreationKit Wiki
Jump to navigation Jump to search
imported>ShadeMe
(update on esl limits)
imported>Qazaaq
Line 1: Line 1:
'''File Extension:''' <code>.esm</code>, <code>.esp</code>, <code>.esl</code>, <code>.fos</code>
'''File Extension:''' <code>.esm</code>, <code>.esl</code>, <code>.esp</code>, <code>.fos</code>, <code>.fid</code>, <code>.fud</code>, <code>.fvd</code>, <code>.ccc</code>


A '''Data File''' acts as a database of all of the data for the world, including object data, dialogue, gameplay settings, object placements, AI settings, landscape, cells, etc.  
A '''Data File''' acts as a database of all of the data for the world, including object data, dialogue, gameplay settings, object placements, AI settings, landscape, [[Cell]]s, etc.
There are several kinds of files associated with data files.
Masters, Light Masters, and Plugins are largely identical in format, but have some important distinctions in practice.
Other files associated with game data include [[#Game Save|Game Saves]], [[Version Control]] meta, and temporary files generated by the [[Creation Kit]].
 
== Master ==
<code>.esm</code>
<BR>
The [[Creation Kit]] will not create master files without additional enabling the Creation Kit's built in [[Version Control]] system.
The merging process takes the interim ESP data and merges into a master ESM file to be used by the collaborators.
 
==== Removing ESM Files ====
While ESP files can usually be removed from gameplay simply by de-activating them, ESM files cannot be removed in this way.
In short the [[Creation Engine]] refuses to remove an ESM from the save game.
However, this can easily overcome by modifying the savegame to think that the ESM was actually an ESP -- at which point the game engine will happily remove it.
(See Wrye Bash: Disabling Masters.)


==Files==
There are three types of data files -- Master Files and Plugin Files, and new as of the 1.10 Update, Light Master files.
Master files (normal and Light) and plugins are largely identical in format, but have some important distinctions in practice.


===Master===  
== Light Master ==
<code>.esm</code>
<code>.esl</code>
<BR>
[[Fallout 4|Fallout 4 v1.10+]]
<BR>
The ''Light Master'' (.esl) file is a new optimized file format introduced in the v1.10 update.
Using ''Light Master'' files will allow more plugins to be loaded than the previously possible due to the 8-bit plugin ID limit of '''255'''.
A ''Light Master'' does this by using more of the ''Form ID'' range for load order, while also sacrificing it's maximum capacity for form records.
 
A ''Light Master'' file cannot be edited in the [[Creation Kit]].
Active development should be done in an normal <code>*.esp</code> plugin file.
 
==== Light Master Notes ====
*''Light Master'' files released by Bethesda Game Studios ([[Creation Club]] releases) are flagged as master files and always load immediately after the official Bethesda masters, in the order specified by the game's <code>.ccc</code> file. These .esl files cannot be deactivated.
*''Light Master'' files released by modders are flagged the same as ESP files, but these '''also''' load with the master files, in the order specified in <code>plugins.txt</code>. These .esl files will need to be activated before the game will load them.
*''Light Master'' files created by modders cannot use standard ESP files as masters or this will cause the ESP file to also load before the .esl in the master files section. This is new behavior for the game engine as of Skyrim Special Edition v1.5.3.
** Any dependencies of the normal plugin a light master depends on are '''not''' loaded early.
** Creating light masters that are dependent on normal plugins is highly discouraged.
 
==== Light Master Budgets ====
A ''Light Master'' file has a budget of no more than 2,048 forms, with Form IDs <code>0x800</code> to <code>0xFFF</code> available for use.
A absolute maximum of 4096 ESL files can be loaded by the engine at once, but in practice this is difficult to achieve.
331 plugins can be loaded that each contain 2048 [[ALCH]] records, but 970+ plugins can be loaded that each contain one [[CELL]] record and 2047 [[REFR]] records.
There may also be a file handle limit, and a reference limit that prevent 4096 ESL files from being loaded.
 
==== Light Master Runtime ====
''Light Master'' files are merged into plugin slot <code>0xFE</code> at runtime.
Their runtime Form ID is <code>0xFE[000](Hex Load Order)[000](Original Form ID)</code>.
 
Form <code>0xFA0</code> from the 10th loaded ESL file would be <code>0xFE00AFA0</code> at runtime.
From the 200th ESL file, it would be <code>0xFE0C8FA0</code>.
 
==== Light Master Publishing ====
To publish an (.esp) plugin as an (.esl) light master, load your ESP file, and under ''File'', select ''Convert Active File to Light Master''.
It is good practice to run ''Compact Active File'' Form IDs to optimize your Form IDs within the 2,048 form budget before converting.
Compacting Form IDs does also change Form IDs in the ESP file, so be sure do this before converting to ESL, and be sure to save the ESP file afterwards.
 
Do not compact Form IDs on an ESP file you have already released and are updating.
Because Form IDs change, you essentially create a new plugin when compacting.
This new plugin will not be compatible with saves using the old version.
 
==== Light Master Updating ====
Updating ESL files has more restrictions than updating ESP files, but not many.
Because of the 2,048 Form ID limit, it is important to be careful when deleting forms not to reuse Form IDs.
A saved weapon form could be deleted in your plugin, then have an armor form take the Form ID when compacted.


*The [[Creation Kit]] will not create master files.
Be sure to save ESP files after running ''Compact Active File'', because otherwise compacting again with new forms might give different Form IDs for forms already in a game save.
** NOTE: An exception to this rule is when using the Creation Kit's built in version control system. The merging process takes the interim esp data and merges into a master file to be used by the collaborators. See this page for details [[Version Control and Multiple Contributor Collaboration]]
As long as development is done on compacted ESP files that match released ESL files, forms can be added under the 2,048 form limit, and existing forms can be edited without issue.


===Plugin===
==== Light Master to Plugin Conversion ====
<code>.esp</code>
While not directly supported, an ESL can be converted back into an ESP for direct editing.


*A plugin ESP file created in the [[Creation Kit]] may only rely on data from itself and any master ESM files.
As of '''Update v1.10+''', simply rename the ESL file extension to <code>*.esp</code>.
**Plugins are '''not''' required to contain any data.
With the renamed file extension, the [[Creation Kit]] will load the ESL as a standard ESP plugin.
**Plugins are '''not''' required to have a master if they are empty or add '''new''' content.
When ''Active Plugin'' is saved, the [[Creation Kit]] will complete the conversion by removing the master header flag.
**Plugins can only refer directly to data from a master file, though they can refer to multiple master files.
The ESl is now converted into an ESP plugin file.
**Plugins can only refer to information in another plugin indirectly by using scripts.
*It's also possible to create plugins that depend on other plugins using third-party tools.


===Light Master===
The ''Convert Active File to Light Master'' command may be used to publish the file as an ESL light master again.
<code>.esl</code>


*Light Master (.esl) files are a new optimized file format, introduced in Fallout 4 v1.10. They behave like a normal plugin, except for the following:
==== Light Master to Plugin Manual Conversion ====
**.esl files cannot be edited in the [[Creation Kit]]. Active development should be done in an normal .esp file.
Before '''Update v1.10''', manually converting a light master to a plugin was not supported by the [[Creation Kit]].
**.esl files allow more plugins to be loaded than the previous 8-bit plugin ID limit (255), by using more of the Form ID for their load order.
To manually convert a light master to a plugin, do the following.
**.esl files have a budget of no more than 2,048 forms, with Form IDs <code>0x800</code> to <code>0xFFF</code> available for use.
# Rename the file back to <code>.esp</code> and open it for editing with [[xEdit]],
***A absolute maximum of 4096 .esl files can be loaded by the engine at once, but in practice this is difficult to achieve.
# In the '''Header''' section, edit the flags to trigger a resave. Simply choosing the '''ESM''' flag on and off again will suffice.
***331 plugins can be loaded that each contain 2048 ALCH records, but 970+ plugins can be loaded that each contain one CELL record and 2047 REFR records.
# Save the modified plugin. It will then be a binary identical to the original <code>*.esp</code> it was created from.
***There may also be a file handle limit, and a reference limit that prevent 4096 .esl files from being loaded.
**.esl files are merged into plugin slot <code>0xFE</code> at runtime. Their runtime Form ID is <code>0xFE[000](Hex Load Order)[000](Original Form ID)</code>.
***Form <code>0xFA0</code> from the 10th loaded .esl file would be <code>0xFE00AFA0</code> at runtime. From the 200th .esl file, it would be <code>0xFE0C8FA0</code>.


*To convert a normal plugin (.esp) to a light (.esl) master, load your .esp file, and under File, select Convert Active File to Light Master.
*It is good practice to run Compact Active File Form IDs to optimize your Form IDs within the 2,048 form budget before converting.
**Compacting Form IDs does also change Form IDs in the .esp file, so be sure do this before converting to .esl, and be sure to save the .esp file afterwards.
**Do not compact Form IDs on an .esp file you have already released and are updating. Because Form IDs change, you essentially create a new plugin when compacting, and this new plugin will not be compatible with saves using the old version.


*Updating .esl files has more restrictions than updating .esp files, but not many.
== Plugin ==
**Because of the 2,048 Form ID limit, it is important to be careful when deleting forms not to reuse Form IDs.
<code>.esp</code>
***A saved weapon form could be deleted in your plugin, then have an armor form take the FormID when compacted.
<BR>
**Again, be sure to save compacted .esp files, because otherwise compacting again with new forms might give different Form IDs for forms already in a game save.
A plugin ESP file created in the [[Creation Kit]] may only rely on data from itself and any master ESM files.
**As long as development is done on compacted .esp files that match released .esl files, forms can be added under the 2,048 form limit, and existing forms can be edited without issue.
It's also possible to create plugins that depend on other plugins using third-party tools.
*Plugins are '''not''' required to contain any data.
*Plugins are '''not''' required to have a master if they are empty or add '''new''' content.
*Plugins can only refer directly to data from a master file, though they can refer to multiple master files.
*Plugins can only refer to information in another plugin indirectly by using scripts.


*.esl Files released by Bethesda Game Studios (Creation Club releases) are flagged as master files and always load immediately after the official Bethesda masters, in the order specified by the game's <code>.ccc</code> file. These .esl files cannot be deactivated.
*.esl Files released by modders are flagged the same as .esp files, but these '''also''' load with the master files, in the order specified in plugins.txt. These .esl files will need to be activated before the game will load them.
*.esl Files created by modders cannot use standard .esp files as masters or this will cause the .esp file to also load before the .esl in the master files section. This is new behavior for the game engine as of Skyrim Special Edition v1.5.3.
** Any dependencies of the normal plugin a light master depends on are '''not''' loaded early.
** Creating light masters that are dependent on normal plugins is highly discouraged.


===Save===
== Game Save ==
<code>.fos</code>
<code>.fos</code>
<BR>
A Fallout Save (.fos) is used to restore the games state between sessions.
The [[Creation Kit]] cannot create, edit, or view save files.
== Version Control ==
<code>.fid</code>, <code>.fud</code>, <code>.fvd</code>
<BR>
The FID, FUD, and FVD files are generated by the [[Version Control]] system.
These files are used by the [[Creation Kit]] to track and merge changes between file versions.


*A Fallout Save (.fos) is used to restore the games state between sessions.
*The [[Creation Kit]] will not create, edit or view save files.


==Tooling==
= Tooling =
*The [[Creation Kit]] is the primary tooling for working with data files.
*The [[Creation Kit]] is the primary tooling for working with data files.
*[[xEdit]]
*[[xEdit]]
*[[Wrye Bash]]
*There are other community software projects that provided tooling for working with data files.
*There are other community software projects that provided tooling for working with data files.


==Notes==
 
= Notes =
*By tradition, the file extensions stand for '''E'''lder '''S'''crolls '''M'''aster and '''E'''lder '''S'''crolls '''P'''lugin.
*By tradition, the file extensions stand for '''E'''lder '''S'''crolls '''M'''aster and '''E'''lder '''S'''crolls '''P'''lugin.
*While not directly supported, a .esl can be converted back to a .esp for editing in one of the following ways.
*#Rename the file back to .esp. Open it for editing with FO4Edit, and in the Header section edit the flags to trigger a resave. Simply choosing the ESM flag on and off again will suffice. Then save. It will then be binary identical to the original .esp it was created from.
*#With the current version of the [[Creation Kit]] (1.10 at time of writing) you can also simply rename the .esl to .esp and open it. The Creation Kit will identify it as a plugin ignoring the header flag and allow editing. When You are finished you just save like normal. If you save as a normal .esp the header is automatically converted back to that of a standard plugin. You may use the convert to .esl function to publish the file as a .esl master again. This behavior may change in the future.


== See Also ==
 
*[[Version Control and Multiple Contributor Collaboration]]
= See Also =
*[[:Category:Game Files|Game Files Category]]
*[[Version Control]]
*[[Game Files]]
 
== Links ==
* https://cs.elderscrolls.com/index.php?title=Esp_vs._Esm
* https://cs.elderscrolls.com/index.php?title=De-Isolation_Tutorial




[[Category:Editor Reference]]
[[Category:Editor Reference]]
[[Category:Game Files]]
[[Category:Game Files]]

Revision as of 17:19, 2 September 2019

File Extension: .esm, .esl, .esp, .fos, .fid, .fud, .fvd, .ccc

A Data File acts as a database of all of the data for the world, including object data, dialogue, gameplay settings, object placements, AI settings, landscape, Cells, etc. There are several kinds of files associated with data files. Masters, Light Masters, and Plugins are largely identical in format, but have some important distinctions in practice. Other files associated with game data include Game Saves, Version Control meta, and temporary files generated by the Creation Kit.

Master

.esm
The Creation Kit will not create master files without additional enabling the Creation Kit's built in Version Control system. The merging process takes the interim ESP data and merges into a master ESM file to be used by the collaborators.

Removing ESM Files

While ESP files can usually be removed from gameplay simply by de-activating them, ESM files cannot be removed in this way. In short the Creation Engine refuses to remove an ESM from the save game. However, this can easily overcome by modifying the savegame to think that the ESM was actually an ESP -- at which point the game engine will happily remove it. (See Wrye Bash: Disabling Masters.)


Light Master

.esl
Fallout 4 v1.10+
The Light Master (.esl) file is a new optimized file format introduced in the v1.10 update. Using Light Master files will allow more plugins to be loaded than the previously possible due to the 8-bit plugin ID limit of 255. A Light Master does this by using more of the Form ID range for load order, while also sacrificing it's maximum capacity for form records.

A Light Master file cannot be edited in the Creation Kit. Active development should be done in an normal *.esp plugin file.

Light Master Notes

  • Light Master files released by Bethesda Game Studios (Creation Club releases) are flagged as master files and always load immediately after the official Bethesda masters, in the order specified by the game's .ccc file. These .esl files cannot be deactivated.
  • Light Master files released by modders are flagged the same as ESP files, but these also load with the master files, in the order specified in plugins.txt. These .esl files will need to be activated before the game will load them.
  • Light Master files created by modders cannot use standard ESP files as masters or this will cause the ESP file to also load before the .esl in the master files section. This is new behavior for the game engine as of Skyrim Special Edition v1.5.3.
    • Any dependencies of the normal plugin a light master depends on are not loaded early.
    • Creating light masters that are dependent on normal plugins is highly discouraged.

Light Master Budgets

A Light Master file has a budget of no more than 2,048 forms, with Form IDs 0x800 to 0xFFF available for use. A absolute maximum of 4096 ESL files can be loaded by the engine at once, but in practice this is difficult to achieve. 331 plugins can be loaded that each contain 2048 ALCH records, but 970+ plugins can be loaded that each contain one CELL record and 2047 REFR records. There may also be a file handle limit, and a reference limit that prevent 4096 ESL files from being loaded.

Light Master Runtime

Light Master files are merged into plugin slot 0xFE at runtime. Their runtime Form ID is 0xFE[000](Hex Load Order)[000](Original Form ID).

Form 0xFA0 from the 10th loaded ESL file would be 0xFE00AFA0 at runtime. From the 200th ESL file, it would be 0xFE0C8FA0.

Light Master Publishing

To publish an (.esp) plugin as an (.esl) light master, load your ESP file, and under File, select Convert Active File to Light Master. It is good practice to run Compact Active File Form IDs to optimize your Form IDs within the 2,048 form budget before converting. Compacting Form IDs does also change Form IDs in the ESP file, so be sure do this before converting to ESL, and be sure to save the ESP file afterwards.

Do not compact Form IDs on an ESP file you have already released and are updating. Because Form IDs change, you essentially create a new plugin when compacting. This new plugin will not be compatible with saves using the old version.

Light Master Updating

Updating ESL files has more restrictions than updating ESP files, but not many. Because of the 2,048 Form ID limit, it is important to be careful when deleting forms not to reuse Form IDs. A saved weapon form could be deleted in your plugin, then have an armor form take the Form ID when compacted.

Be sure to save ESP files after running Compact Active File, because otherwise compacting again with new forms might give different Form IDs for forms already in a game save. As long as development is done on compacted ESP files that match released ESL files, forms can be added under the 2,048 form limit, and existing forms can be edited without issue.

Light Master to Plugin Conversion

While not directly supported, an ESL can be converted back into an ESP for direct editing.

As of Update v1.10+, simply rename the ESL file extension to *.esp. With the renamed file extension, the Creation Kit will load the ESL as a standard ESP plugin. When Active Plugin is saved, the Creation Kit will complete the conversion by removing the master header flag. The ESl is now converted into an ESP plugin file.

The Convert Active File to Light Master command may be used to publish the file as an ESL light master again.

Light Master to Plugin Manual Conversion

Before Update v1.10, manually converting a light master to a plugin was not supported by the Creation Kit. To manually convert a light master to a plugin, do the following.

  1. Rename the file back to .esp and open it for editing with xEdit,
  2. In the Header section, edit the flags to trigger a resave. Simply choosing the ESM flag on and off again will suffice.
  3. Save the modified plugin. It will then be a binary identical to the original *.esp it was created from.


Plugin

.esp
A plugin ESP file created in the Creation Kit may only rely on data from itself and any master ESM files. It's also possible to create plugins that depend on other plugins using third-party tools.

  • Plugins are not required to contain any data.
  • Plugins are not required to have a master if they are empty or add new content.
  • Plugins can only refer directly to data from a master file, though they can refer to multiple master files.
  • Plugins can only refer to information in another plugin indirectly by using scripts.


Game Save

.fos
A Fallout Save (.fos) is used to restore the games state between sessions. The Creation Kit cannot create, edit, or view save files.


Version Control

.fid, .fud, .fvd
The FID, FUD, and FVD files are generated by the Version Control system. These files are used by the Creation Kit to track and merge changes between file versions.


Tooling

  • The Creation Kit is the primary tooling for working with data files.
  • xEdit
  • Wrye Bash
  • There are other community software projects that provided tooling for working with data files.


Notes

  • By tradition, the file extensions stand for Elder Scrolls Master and Elder Scrolls Plugin.


See Also

Links