Difference between revisions of "Version Control"

From the Fallout4 CreationKit Wiki
Jump to navigation Jump to search
imported>Qazaaq
(added see also)
imported>Qazaaq
m
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
What is Version Control?
<BR>
The [[Version Control]] system allows multiple developers to work simultaneously on a single [[Data File]].
The [[Version Control]] system allows multiple developers to work simultaneously on a single [[Data File]].
The [[Creation Kit]] does this by merging the contents of a multiple child plugins into a single master plugin.
During production of [[Fallout 4]], the team at Bethesda used ''version control'' as a way to manage multiple artists, producers, designers, and programmers all working together on the same content.
Using the version control system is essential to working on team projects.
The [[Creation Kit]] ''version control'' system is also capable of providing intergrations with conventional source control managers such as [[Perforce]] or [[Git]].
This page will document the process of mod collaboration with multiple contributors.
The primary way the [[Creation Kit]] provides ''version control'' is by merging the contents of multiple child plugins into a single master plugin.
This document will not cover how to use external revision control software such as ''Perforce'', or ''Git''.
Using the ''version control'' system is essential to working on team projects.
 
This document will cover mod collaboration with multiple contributors.
This document will '''not''' cover how to use external revision control software such as [[Perforce]], or [[Git]].
This document assumes basic usage knowledge of the Creation Kit.
This document assumes basic usage knowledge of the Creation Kit.


= Setup =
= Setup =
Line 11: Line 15:
This will involve creating directories, network sharing folders, editing [[Initialization File]]s, and creating a master [[Data File]].
This will involve creating directories, network sharing folders, editing [[Initialization File]]s, and creating a master [[Data File]].


=== Understanding Data Files ===
The .esp files are "Bethesda Plugin Data Files"
* This is the standard file type created by the Creation Kit, to save your mod
* It contains all of the asset reference data used in your mod
<br />
The .esm files are "Bethesda Master Data Files"
* This is the file type you'll see used by DLC
* It contains the same information used by .esp files, but it can have references merged into it
<br />
In the context of using version control in the Fallout 4 Creation Kit:
* The .esm master file will be your primary mod file, containing all the level and asset reference data for your mod. When you're ready to distribute your mod, it's the .esm that you'll upload.
* The .esp file will be the file you and your project contributors use locally to make all your changes. After you merge your changes into the .esm file, the .esp won't contain any references
* The first time you save your mod, it will be saved as .esp (you do not have a choice)
* It will become .esm the first time you do a merge to prepare it for multi-contributor collaboration


=== Create Directories ===
=== Create Directories ===
Line 17: Line 37:
* <code>...\steamapps\common\Fallout 4\Merging\VersionBackup</code>
* <code>...\steamapps\common\Fallout 4\Merging\VersionBackup</code>
* <code>...\steamapps\common\Fallout 4\Data\CheckInBackup</code>
* <code>...\steamapps\common\Fallout 4\Data\CheckInBackup</code>


=== Enable Windows Sharing ===
=== Enable Windows Sharing ===
Line 34: Line 53:
* Make note of the '''Network Path''' displayed, you'll need this for the next step.
* Make note of the '''Network Path''' displayed, you'll need this for the next step.


=== Enable Version Control ===
{{Enable Version Control}}


=== Create Master File ===
== First merge and creation of your .esm file ==
The Creation Kit version control system will need a master [[Data File]] to merge child plugins into.
Regardless of the content, once the mod has been saved as an <code>*.esp</code>, you must perform a first merge to create your initial <code>*.esm</code> file.
There are various ways to create new master files with community made software.
Some of these include [[XEdit]], [[Wrye Bash]], or any hexidecial editor.
There are even more software projects that can be found with an internet search.
 
Create a new master plugin.
 
 
=== User Information ===
Use the windows command line to get your ''User Name'' and ''Computer Name''.
You can optionally execute the batch file below to get configuration information.
 
====== VersionControl.bat ======
<source lang="winbatch">
@ECHO OFF
ECHO Use these values to configure the Creation Kit version control system.
ECHO.
ECHO Computer Name: %COMPUTERNAME%
ECHO User Name: %USERNAME%
ECHO.
PAUSE
</Source>
 
====== Example Output ======
<source lang="text">
Use these values to configure the Creation Kit version control system.
 
Computer Name: KS07
User Name: Scrivener07
 
Press any key to continue . . .
</Source>
 
 
=== Initialization Files ===
* Navigate to the <code>...\steamapps\common\Fallout 4</code>, the Fallout 4 directory.
* Open\Create the <code>CreationKitCustom.ini</code> [[Initialization File]].
* Populate the <code>CreationKitCustom.ini</code> with the following:
<source lang="ini">bUseVersionControl=1</source>
 
* Add the following lines (anywhere under the "[General]" heading):
 
<source lang="ini">
SNetwork Path=\\COMP-NAME\Merging\
SNewVersionBackupPath=\\COMP-NAME\Merging\VersionBackup\
SNetworkMasterPath=\\COMP-NAME\Merging\Data\
SLocalBackupPath=Data\CheckInBackup\
</source>
* Note:
*: - There is a space in "SNetwork Path"
*: - Replace "COMP-NAME" with your computer name, displayed on the window in the above screenshot
*: - Do not omit the "\" at the end of the paths above, or this will not work properly
*: [[File:versioncontrol_creationkit.png]]
 
 
* Finally, open "CreationKitPrefs.ini"
* Scroll to the bottom of the file, and add the following lines
<source lang="ini">
[WhoCanMerge]
YOURWINDOWSUSERNAME=1
</source>
* Note:
*: - Replace "YOURWINDOWSUSERNAME" with your actual windows username
*: [[File:versioncontrol_creationkitprefs.png]]
 


= Understanding Data Files =
* Navigate to your <code>...\Steam\steamapps\Common\Fallout 4\Data</code> directory.
<br />
* Copy "Fallout4.esm" to <code>...\Steam\steamapps\Common\Fallout 4\Merging\Data</code>.
.esp files are "Bethesda Plugin Data Files"
** If you don't copy this file, the merge process will error out.
* This is the standard file type created by the Creation Kit, to save your mod
* Next, open your <code>*.esp</code> as the ''Active File''
* It contains all of the asset reference data used in your mod
* After the file has finished loading in the Creation Kit, click "File" -> "Data...", or the open folder icon.
<br />
.esm files are "Bethesda Master Data Files"
* This is the file type you'll see used by DLC
* It contains the same information used by .esp files, but it can have references merged into it
<br />
In the context of using version control in the Fallout 4 Creation Kit:
* The .esm master file will be your primary mod file, containing all the level and asset reference data for your mod. When you're ready to distribute your mod, it's the .esm that you'll upload.
* The .esp file will be the file you and your project contributors use locally to make all your changes. After you merge your changes into the .esm file, the .esp won't contain any references
* The first time you save your mod, it will be saved as .esp (you do not have a choice)
* It will become .esm the first time you do a merge to prepare it for multi-contributor collaboration
 
 
== First merge and creation of your .esm file ==
<br/>
Once you've saved your mod as a .esp (regardless of the content), you must do a first merge to create your initial .esm file:
<br/>
* Navigate to your "Fallout 4\Data" directory
* Copy "Fallout4.esm" to "Fallout 4\Merging\Data"
** If you don't copy this file, the merge process will error out
* Next, open your .esp as the Active File
* After the file has finished loading in the Creation Kit, click "File" -> "Data..." (or the open folder icon)
* Select "Fallout4.esm" in the list, and click "Details..."
* Select "Fallout4.esm" in the list, and click "Details..."
*: [[File:versioncontrol_data_Details.png]]
*: [[File:versioncontrol_data_Details.png]]
* When asked "Do you want to use an empty list to save memory?", click "Yes"
* When asked "Do you want to use an empty list to save memory?", click '''Yes'''.
* You'll now be shown a new popup titled "File Details", but it will have no contents.
* You'll now be shown a new popup titled '''File Details''', but it will have no contents.
** On your keyboard, press Shift+Ctrl+B
** On your keyboard, press {{key press|Shift|Ctrl|B}}.
* You'll be asked if you want to update the Bit Array Files, and warned it can take several minutes. Of course, click "Yes".
* You'll be asked if you want to update the '''Bit Array Files''', and warned it can take several minutes. Of course, click '''Yes'''.
*: [[File:versioncontrol_update_bit_array_file.png]]
*: [[File:versioncontrol_update_bit_array_file.png]]
* When it's finished, you'll see "Updating bit arrays...Done" in the bottom status bar of the main window
* When it's finished, you'll see "Updating bit arrays...Done" in the bottom status bar of the main window.
*: [[File:versioncontrol_update_bit_array_done.png]]
*: [[File:versioncontrol_update_bit_array_done.png]]
<br/>
<br/>
Now you're ready to create your .esm file!
Now you're ready to create your .esm file!
* Ensure the .esp mod file has been opened as the Active File (this should have been done earlier, but do it again if you restarted the Creation Kit)
* Ensure the <code>.esp</code> mod file has been opened as the ''Active File''. This should have been done earlier, but do it again if you restarted the [[Creation Kit]].
* You'll notice that you have a new little Version Control icon from when we enabled it in the Creation Kit. Click it.
* You'll notice that you have a new ''Version Control'' [[File:Version Control Icon.png]] icon on the [[Editor_Interface#Toolbar|Editor Toolbar]]. '''Click it'''.
*: [[File:versioncontrol_button.png]]
*: [[File:versioncontrol_button.png]]
* For this tutorial, I created "MyMod.esp", and I duplicated the "aaaMarkers" Cell from the base game Fallout4.esm
* For this tutorial, I created "MyMod.esp", and I duplicated the "aaaMarkers" Cell from the base game Fallout4.esm
Line 218: Line 156:
<br/>
<br/>
<br/>
<br/>
== Merge process and workflow of the individual team member ==
== Merge process and workflow of the individual team member ==
<br/>
<br/>
Line 267: Line 204:
* [[CreationKit.ini]]
* [[CreationKit.ini]]
* [[CreationKitPrefs.ini]]
* [[CreationKitPrefs.ini]]
* [https://support.microsoft.com/en-us/help/4092694 File sharing over a network in Windows 10].
 
== Links ==
* [https://www.creationkit.com/index.php?title=Version_control Skyrim - Version Control]
* [https://www.creationkit.com/index.php?title=Version_control Skyrim - Version Control]
* [https://www.creationkit.com/index.php?title=Mod_Merging Skyrim - Mod Merging]
* [http://geck.bethsoft.com/index.php?title=Version_Control Fallout 3 & New Vegas - Version Control]
* [https://cs.elderscrolls.com/index.php?title=Version_Control Elder Scrolls III & IV - Version Control]
* [https://support.microsoft.com/en-us/help/4092694 File sharing over a network in Windows 10 at Microsft.com].
* [https://www.perforce.com/ Perforce - Source Control Manager]
* [https://git-scm.com/ Git - Source Control Manager]
* [http://wiki.tesnexus.com/index.php/Version_control_primer Elder Scrolls IV - Version Control Primer at NexusMods.com]
* [https://www.youtube.com/watch?v=6eInvAyMXoI Wheeze's Creation Kit Tutorials - Skyrim Version Control]
* [https://www.youtube.com/watch?v=6eInvAyMXoI Wheeze's Creation Kit Tutorials - Skyrim Version Control]
* [https://www.perforce.com/ Perforce - Source Control Manager]
* [https://git-scm.com/ git - Source Control Manager]




[[Category:Editor Reference]]
[[Category:Editor Reference]]
[[Category:Tutorials]]
[[Category:Tutorials]]

Latest revision as of 00:47, 31 August 2019

What is Version Control?
The Version Control system allows multiple developers to work simultaneously on a single Data File. During production of Fallout 4, the team at Bethesda used version control as a way to manage multiple artists, producers, designers, and programmers all working together on the same content. The Creation Kit version control system is also capable of providing intergrations with conventional source control managers such as Perforce or Git. The primary way the Creation Kit provides version control is by merging the contents of multiple child plugins into a single master plugin. Using the version control system is essential to working on team projects.

This document will cover mod collaboration with multiple contributors. This document will not cover how to use external revision control software such as Perforce, or Git. This document assumes basic usage knowledge of the Creation Kit.

Setup[edit | edit source]

To begin, configure the Creation Kit to enable version control. This will involve creating directories, network sharing folders, editing Initialization Files, and creating a master Data File.

Understanding Data Files[edit | edit source]

The .esp files are "Bethesda Plugin Data Files"

  • This is the standard file type created by the Creation Kit, to save your mod
  • It contains all of the asset reference data used in your mod


The .esm files are "Bethesda Master Data Files"

  • This is the file type you'll see used by DLC
  • It contains the same information used by .esp files, but it can have references merged into it


In the context of using version control in the Fallout 4 Creation Kit:

  • The .esm master file will be your primary mod file, containing all the level and asset reference data for your mod. When you're ready to distribute your mod, it's the .esm that you'll upload.
  • The .esp file will be the file you and your project contributors use locally to make all your changes. After you merge your changes into the .esm file, the .esp won't contain any references
  • The first time you save your mod, it will be saved as .esp (you do not have a choice)
  • It will become .esm the first time you do a merge to prepare it for multi-contributor collaboration

Create Directories[edit | edit source]

Navigate to the Fallout 4 installation directory and create the following new folders.

  • ...\steamapps\common\Fallout 4\Merging\Data
  • ...\steamapps\common\Fallout 4\Merging\VersionBackup
  • ...\steamapps\common\Fallout 4\Data\CheckInBackup

Enable Windows Sharing[edit | edit source]

Enable Windows File Sharing on the following two folders.

  • ...\steamapps\common\Fallout 4\Merging
  • ...\steamapps\common\Fallout 4\Data\CheckInBackup

Configure sharing for CheckInBackup, and then repeat these steps for the Merging folder. There is no need to share any of the Merging sub-folders.

  • Right click on the CheckInBackup folder and select Properties.
  • Select the Sharing tab -> click Advanced Sharing....
  • Click Share this folder, and then OK.
  • Back on the Properties windows, click the Share... button.
    SharePropertiesTab.png
  • You won't actually be sharing your computer with anyone else, so you don't have to add any additional users - just click Share -> Done.
  • Make note of the Network Path displayed, you'll need this for the next step.

Enable Version Control[edit | edit source]

The Version Control system allows multiple developers to work simultaneously on a single Data File. By default, Version Control features are disabled within the Creation Kit.

The following Initialization File settings will need to be configured to use Version Control.

CreationKitCustom.ini[edit source]

Add or change the following under the [General] section heading of CreationKitCustom.ini. Replace COMPUTERNAME with your Windows Computer Name. Note the space in sNetwork Path and do not omit the leading or trailing path seperators.

[General]
bUseVersionControl=1
sNetwork Path=\\COMPUTERNAME\Merging\
sNewVersionBackupPath=\\COMPUTERNAME\Merging\VersionBackup\
sNetworkMasterPath=\\COMPUTERNAME\Merging\Data\
sLocalBackupPath=Data\CheckInBackup\
CreationKitPrefs.ini[edit source]

Add or change the following under the [WhoCanMerge] section heading of CreationKitPrefs.ini. Replace USERNAME with your Windows User Name.

[WhoCanMerge]
USERNAME=1

First merge and creation of your .esm file[edit | edit source]

Regardless of the content, once the mod has been saved as an *.esp, you must perform a first merge to create your initial *.esm file.

  • Navigate to your ...\Steam\steamapps\Common\Fallout 4\Data directory.
  • Copy "Fallout4.esm" to ...\Steam\steamapps\Common\Fallout 4\Merging\Data.
    • If you don't copy this file, the merge process will error out.
  • Next, open your *.esp as the Active File
  • After the file has finished loading in the Creation Kit, click "File" -> "Data...", or the open folder icon.
  • Select "Fallout4.esm" in the list, and click "Details..."
    Versioncontrol data Details.png
  • When asked "Do you want to use an empty list to save memory?", click Yes.
  • You'll now be shown a new popup titled File Details, but it will have no contents.
    • On your keyboard, press ⇧ Shift+Ctrl+B.
  • You'll be asked if you want to update the Bit Array Files, and warned it can take several minutes. Of course, click Yes.
    Versioncontrol update bit array file.png
  • When it's finished, you'll see "Updating bit arrays...Done" in the bottom status bar of the main window.
    Versioncontrol update bit array done.png


Now you're ready to create your .esm file!

  • Ensure the .esp mod file has been opened as the Active File. This should have been done earlier, but do it again if you restarted the Creation Kit.
  • You'll notice that you have a new Version Control Version Control Icon.png icon on the Editor Toolbar. Click it.
    Versioncontrol button.png
  • For this tutorial, I created "MyMod.esp", and I duplicated the "aaaMarkers" Cell from the base game Fallout4.esm
    Versioncontrol changedreferences.png
  • In the bottom left, the "Altered" View will show all of the changes detected between the Fallout4.esm file, and MyMod.esp
  • Select all the files in the list, and click "Check Out". This will deselect them.
  • Select all the files in the list, and click "Check In". This will open a comment box. You can ignore this for the first time .esm creation, and just click "OK".
  • Next it will open a save dialogue, defaulting to \\COMP-NAME\Merging\Data
    • Save your plugin using any name (presumably the same name as your .esp, but notice the file extension is .esm in the save dialogue)
    • Do NOT attempt to overwrite Fallout4.esm
    Versioncontrol esm save dialogue.png
    • When asked "Do you wish to select a Check In Data file to use for assign forms new IDs on check in?", select "No"
  • The results are output into the normal Data folder
    • You can see that after a merge is performed, the filesize of the .esp is now 1KB, because the contents have been transferred to the newly created .esm
    Versioncontrol sizechange.png

Workflow for multiple team members contributing to the same .ESM file[edit | edit source]


As mentioned at the top of the page, this document will not explain how to use external revision control software such as Perforce or Git, but now we will discuss the workflow for multiple team members contributing to the same project while using such software.
Think of the .esm as the one true source for what your mod is. When you want to make changes to it, you make those changes through the .esp file. The .esp file is essentially a changelist for your local edits.
Let's say a room already exists in your .esm master file:

  • When you modify a single wall in that room and save, only the modifications for that wall get saved to the .esp -- the rest of the room is not in the .esp, because it remains unchanged in the .esm file.
  • When you merge the .esp into the .esm, the Creation links the IDs of those changed assets, to the assets in the .esm that share the same IDs.
  • When you add things to the .esp that don't exist in the .esm, the Creation Kit knows those are new because no assets in the .esm share the same ID. So in the merge, those new asset references are added to the .esm.
  • After you merge your changes into the .esm, the .esp is emptied out - there are no more changes left in your .esp, because they are now an integrated part of your .esm as the one true source.



There are a few things to remember with multiple team members working on the same master file.

  • When you initiate a merge, make sure the latest .esm file from the repository is in the Merging\Data folder, not the regular folder.
  • Merges into the .esm by different people must be done one a time.
    • If two people were to make local edits to the mod, and then merge into their local .esm at the same time, work will be lost when both people go and upload the latest master file to the repository.
    • Revision control software usually has their own tools to merge files like this together, but depending on what the changes are (especially changes to new custom content) that can be difficult, and is sometimes just not possible when two people's changes collide.
    • Luckily the backup directories we created way at the top of this page are used by the Creation Kit to backup your .esp file. If a team member was to merge into a local .esm out of turn, their .esp is still in tact in the backup folder, and they can step back in line to merge in the proper queue.


  • Even when properly merging changes one after another - using the last person's updated .esm to initiate your next merge - it's important that team members aren't working on the same assets, thus simply overriding the previous person's change.
    • As an example:
      • Let's say that two people decided to move already existing architecture assets in Layer_7
      • User one moves a room down because they want to add an elevator, and they save these changes to their .esp
      • User two moves the room backwards because they want to add a hallway that leads to the room, and they save these changes to their .esp
      • User one correctly communicates his intent to merge into the .esm
        • User one downloads the latest version of the .esm from the online repository, and initiates the merge process in the Creation Kit
        • User one uploads the new .esm to the online repository
      • User two correctly waits for user one to finish
        • User two downloads the latest .esm that user one just finished uploading, and initiates the merge process in the Creation Kit
        • User two uploads the new .esm to the online repository
    • Uh oh! User two, though following the correct queue for merging changes, just blew away user one's changes. It's as if the room was never moved down, but the new elevator to nowhere probably looks great.


This may seem like an obvious example of where merging can go wrong, but there are many different types of content, assets, scripts, etc... that may be less obvious. Always keep this in mind, and promote frequent communication on your team.

Here's a diagram demonstrating the proper workflow for multiple team members contributing to the same mod.
Versioncontrol workflowdiagram.png

  • User 1 downloads version 1 of the master file
  • User 1 modifies Layer 1


  • User 2 also downloads version 1 of the master file
  • User 2 modifies Layer 2
    • This is okay, because User 1 and User 2 have communicated and understand the scope and direction of each other's changes. They know what sections of the level the other is modifying, and their changes will not collide.


  • User 1 merges his modifications of Layer 1 into the .esm file
  • User 1 uploads version 2 of the .esm to the repository


  • User 2 waits for User 1 to finish his merge.
  • User 2 downloads version 2 of the master file, to ensure it retains all the changes that User 1 just made
  • User 2 merges his modification of Layer 2 into the new .esm file
  • User 2 uploads version 3 of the .esm to the repository


  • User 3 is also in communication with the other users, and he also needs to modify Layer 1
  • User 3 must wait until User 1 is finished before he starts modifying Layer 1, or their changes will collide
  • User 3 downloads version 3 of the master file
  • User 3 modifies Layer 1, which contains the modifications User 1 already made
  • User 3 merges his modifications of Layer 1 into the .esm file
  • User 3 uploads version 4 of the .esm to the repository



Merge process and workflow of the individual team member[edit | edit source]


  • Download the latest .esm file from the repository, to both the \Fallout 4\Data\ folder, AND \Merging\Data\.
  • Open the Creation Kit, and begin making your changes to the .esp file.
  • When you're finished your changes to the .esp, save, and close down the Creation Kit.


  • Download the latest .esm file from the repository, to the \Merging\Data\ folder.
    • A team member could have uploaded a new version while you were working, so don't skip this step. Always ensure you merge into the latest version.
    • This is why team communication is so important. You need to know what your team members plan to work on, and they need to know what you plan to work on, so that no changes collide.
    • The merge is not done from \Fallout 4\Data\, so ensure this is your final step before initiating your merge.
  • With the latest version of the .esm in \Merging\Data\, re-open the Creation Kit.
  • Open the .esp as the Active File
    • Do not try to skip opening the file, the merge won't work without opening it first
  • Once the file is finished opening, click "File" -> "Data..."
  • Click on your mod .esm file, and then "Details".
    Versioncontrol mergeprep details.png
  • When asked "Do you want to use an empty list to save memory?", click "Yes"
  • You'll now be shown a new popup titled "File Details", but it will have no contents.
    • On your keyboard, press Shift+Ctrl+B
  • You'll be asked if you want to update the Bit Array Files, and warned it can take several minutes. Click "Yes".
    Versioncontrol update bit array file.png
  • When it's finished, you'll see "Updating bit arrays...Done" in the bottom status bar of the main window
    Versioncontrol update bit array done.png


  • With the bit array updated, you're ready to merge!
  • Cancel out of the Data window, you don't need to re-open the .esp, because you already opened it.
  • Now click the Version Control button
    Versioncontrol button.png
  • Select all the entries in the list, and click "Check Out"
  • Select all the entries again, and click "Check In"
  • Once you regain control of the window, your merge is complete!


  • The new .esm is in \Merging\Data\. Upload that masterpiece back to the repository!


Good luck!

Notes[edit | edit source]

  • Information contained herein was taken from the Skyrim page, and amended after testing in the Fallout 4 Creation Kit.

See Also[edit | edit source]

Links[edit | edit source]