Getting the necessary data¶
WARNING: As of the latest updates, the acquisition of data can be done easily running:
from autumn.scripts.create_data_tree import main_build_file_tree
main_build_file_tree()
If you want to get the data yourself, go ahead and follow the next steps.
EUROSTATS data¶
In order to be able to perform the aggregation operations and geographical labeling in this tool, geographical information is needed. This information is provided by EuroGraphics through the Eurostat website for non commercial uses. Given unknown restrictions on the redistribution of this data it was decided to keep it out of the repository. The user will need to download it by themselves and put them in the folder. Here are the detailed instructions for doing so:
First, go to the data website found the eurostat geodata website
and download the 2021 SHP data in the 1:1 Million resolution by clicking on it. Once you have the data, extract and unpack the following files into the Data\CostPotentialCurves\Input\NUTS folder:
NUTS_RG_01M_2021_4326_LEVL_0
NUTS_RG_01M_2021_4326_LEVL_1
NUTS_RG_01M_2021_4326_LEVL_2
NUTS_RG_01M_2021_4326_LEVL_3
OPSD data¶
There is two datasets from the Open Energy System that are necessary for the correct functioning of the tool: the european conventional power plant dataset and the german one, both of them can be found in the OPSD website
Download the csv files and put them in: “Data\GeographicalDataHomogenization\Input”
FRESNA data¶
The power plant matching data will be downloaded automatically if you don’t have it installed, just make sure you have installed their api using pip.
pip install powerplantmatching
Industrial Data¶
If you don’t have your own industrial data please email the authors to see options to obtain your own.
Scenario Data¶
The scenario development is done using the Global Energy and Climate Outlook from the European Union. The data is availible in their website You can download the necessary data by going to Energy>Production then clicking the download button at the bottom right.
Once you have downloaded the data, just place it in the Data/scenarios/input directory.
Index data¶
All the index values have associated paths in the config file, please locate the files in the corresponding paths with the desired names or change the desired path, this last option is not recommended to avoid undesirable results
CEPCI¶
The Repository includes a dummy file that contains the CEPCI structure, this is to ease the running of the software but the outputs of the framework won’t be valid until you get proper data.
Directory: CaptureCostHarmonization/input/indexes/CEPCI.csv
Coal¶
The coal cost index was calculated based on the World Bank open data found in their website there is special instructions to download these values, download the zip file at charts and data files and extract the annual commodity prices file, from there copy the values of the coal prices into a separate csv file and store it in the corresponding folder.
Directory: CaptureCostHarmonization/input/indexes/coal_index_WB.csv
Natural Gas¶
For the natural gas price developments the Henry Hub Index is used. This can be found reported in the Energy Information Administration. The used values are an average of the yearly reports.
Directory: CaptureCostHarmonization/input/indexes/natural_gas_hh.csv
IHS Indexes¶
The energy cost and technology indexes were estimated based on information published in the IHS Upstream Cost Indexes these values can’t be extracted directly so tabulate them yourself or send the authors an email. By default the tool uses CEPCI for all transformations so doing this is not absolutely necessary.
Directory: CaptureCostHarmonization/input/indexes/IHS.csv
Current Fuel cost¶
The values for the fuel costs are obtained from the U.S. Energy Information Administration and other similar sources, the emission factors are matched using this report and are collected in the followning json file, save it and place it in the CaptureCostHarmonization/input/fuel_data.json directory. But you don’t have stick to these sources, use the following structure to assign values to the fuels:
{
"Illinois_6_ton": {
"Type": "hard_coal",
"HHV_GJ": 25.35,
"LHV_GJ": 24.12,
"Cost_2019_USD" : 34.473,
"Cost_2020_USD" : 31.29,
"Emission_Factor_KG_KJ" : 0.000094
},
"powder_river_basin_ton": {
"Type": "lignite",
"HHV_GJ": 11.86,
"LHV_GJ": 10.07,
"Cost_2019_USD" : 10.43,
"Cost_2020_USD" : 10.43,
"Emission_Factor_KG_KJ" : 0.000110
},
"natural_gas_m3": {
"Type": "natural_gas",
"HHV_GJ": 40,
"LHV_GJ": 36,
"Cost_2019_USD":84.3,
"Cost_2020_USD": 62.26,
"Emission_Factor_KG_KJ" : 0.000056
},
"black_liqor_ton": {
"Type": "bioenergy",
"HHV_GJ": 21,
"LHV_GJ": 19.3,
"Cost_2019_USD": 63.0,
"Cost_2020_USD": 63.0,
"Emission_Factor_KG_KJ" : 0.000071
}
}
If you think you have more realiable values modify the files in the index folder and then assign your new file to the corresponding entry in the config file. You will need to assign the column names in the “cost_transformation_functions.py” to match the ones in your new index. If your index is not aggregated by year, add it to the aggregated index dictionary in said file, otherwise add it to the basic index dictionary.