SecurEnergy EU · Member State geography
EU Member State Energy Security GIS
Explore historical country indicators and generated Baseline, IMEC, and Piano Mattei forecast paths on the project’s GISCO Member State boundaries.
Loading GIS data
Loading EU Member State boundaries and indicator data...
How to read this map
Geography, direction, and period
Choose a year, scenario, and metric. Historical years show observed country values; forecast years show the selected Baseline, IMEC, or Piano Mattei path. Risk metrics use yellow-to-red colors, where darker red means higher exposure or intensity. Positive coverage metrics use yellow-to-green colors, where darker green means higher coverage or renewable performance. Map colors share the complete generated metric domain across historical and forecast rows.
Metric guide and direction
GISCO 2024 · 20M boundaries
Member State map
--
LowMidHigh
Select or focus a country to inspect its values. Boundaries use the existing GISCO 2024 EU Member State layer; no generic map geometry is substituted.
Within-slice comparison
Country ranking
-
Selected Member State
Select a country
Select a country on the map or in the ranking to inspect available ESS and renewable indicators.
Country-level values
Selected slice table
ESS4 is not present in the generated country-level GIS dataset. This view displays only the metrics available in the original GIS output.
SecurEnergy EU · Indicator comparison
ESS vs Eurostat Comparison
Compare the project’s GDP-scaled ESS1 oil and gas channels with conventional EU27 import-to-consumption quotients, using the generated historical benchmark series.
Loading benchmark data
Loading generated EU27 comparison series...
How to read this comparison
Trend, not level equivalence
This view compares different indicator designs. ESS oil and gas channels adapt the paper’s ESS1 import-dependency/accessibility logic with GDP in the denominator. Conventional benchmarks are direct EU27 import-to-consumption quotients. Their absolute magnitudes are not intended to be numerically equivalent; compare direction and turning points rather than treating the difference as a dependency percentage.
Gap columns equal conventional quotient minus the corresponding ESS1 channel. Positive values mean the conventional quotient is higher for that year. These are raw differences, not percentage points.
Definitions, source, and validation context
Generated source: -
Latest build: - · Publication status: -
Build-level validation: -
Full formula definitions are centralized in the project Method & Formula Atlas.
GDP-scaled project channels
ESS results
ESS oil and ESS gas use the existing generated formula-channel values; they are not conventional import dependency percentages.
Direct import-to-consumption quotients
Eurostat-style results
The conventional values are stored as direct quotients. They are shown without percent conversion or rebasing.
Precomputed historical benchmark
Comparison table
Displayed values round to three decimals as in the legacy view; the API returns the original generated precision. Gap differences retain their raw quotient units.
SecurEnergy EU · Energy and output
ESS and Industrial Production
Compare ESS1, ESS2, and ESS3 with EU27 industrial production on the generated 2015-based index scale.
Loading comparison data
Loading generated ESS and industrial-production series...
Interpretation
Co-movement, not causality
Read the lines as comparative trends. Lower ESS1 suggests lower modeled fossil-import exposure; lower ESS2 indicates less industrial energy per unit of GDP; higher ESS3 indicates stronger domestic production coverage. A higher industrial-production index means stronger output.
ESS1, ESS2, and ESS3 are each rebased to their own 2015 historical value. Industrial production retains the generated Eurostat 2015=100 index. The comparison shows co-movement only and does not establish causality.
ESS1ESS2ESS3
Observed industryEstimated industry segment
No Additional ProjectIMECPiano Mattei
Scenario dash patterns distinguish the ESS paths; scenario colors distinguish industrial-production overlays.
Definitions and data provenance
ESS1: Import-dependency and accessibility indicator. Lower future paths suggest lower modeled fossil-import exposure.
ESS2: Energy intensity, covering accessibility and affordability. Lower future paths mean less industrial energy is needed per unit of GDP.
ESS3: Domestic production-rate indicator. Higher future paths mean stronger domestic coverage and lower fragility to external supply interruptions.
Black line: Historical industrial-production trend for the EU or the sector shown in that panel.
Dotted black line: Missing recent annual industrial-production values are completed with a simple linear extension and shown as a dotted black line when present in the generated data.
Scenario paths: Baseline, IMEC, and Piano Mattei. For ESS these are forecast paths; for industrial production they are sensitivity overlays relative to a backtest-selected baseline using the scenario assumptions for ESS1, ESS2, ESS3, and digital spillovers.
Formula definitions are centralized in the Method & Formula Atlas.
Source artifacts: -
Latest build: - · Publication status: -
-
EU27 · B-D
Total Industry
EU27 · Eurostat industrial groupings
Industrial Sectors
SecurEnergy EU · Macro trends
Nominal vs Real GDP Trends
Compare EU27 nominal GDP with deflated real GDP on the legacy 2015 = 100 index base, including the generated project scenario paths through 2030.
Loading GDP series
Loading generated nominal and real GDP series...
How to read the comparison
Money value and real output
Nominal GDP includes price-level effects. Real GDP deflates those effects using the project’s 2015=100 deflator. The top panel compares indexed levels, the middle panel shows the nominal-minus-real index gap in index points, and the lower panel compares year-over-year growth.
The index gap is not a percentage-point measure. Forecast nominal GDP follows the selected project baseline plus scenario overlays; forecast real GDP deflates those same nominal paths. Forecast deflator growth after 2024 uses the generated page’s recent-average assumption.
Selected nominal-GDP baseline: -; backtest MAPE -%; direction accuracy -%.
Nominal GDPReal GDP
No Additional ProjectIMECPiano Mattei
Source and methodology
-
Generated source: -
Forecast deflator assumption after 2024: -
Indexed levels
Nominal and real GDP
Price-level separation
Nominal-real index gap
Annual change
Year-over-year GDP growth
Generated summary rows
GDP scenario snapshot
Snapshot values reproduce the legacy latest-history and final-forecast rows. The displayed GDP levels are indices, not million-euro amounts.
SecurEnergy EU · Policy analysis
Policy Scenarios, Shocks, and Sensitivity
Compare generated Baseline, IMEC, and Piano Mattei paths, then inspect the sensitivity, drivers, shock assumptions, and validation evidence behind them.
Loading scenario data
Loading policy scenario artifacts...
Reading guide
Direction before level
Read the historical segment first, then compare the colored forecast paths. Lower ESS1 and ESS2 indicate lower import exposure or energy intensity; higher ESS3 and renewable indicators indicate stronger domestic coverage or renewable performance. ESS4 is direction-normalized to 2015 = 100, where higher means stronger composite security. Scenario paths are exploratory, assumption-based sensitivities, not causal effects or point predictions.
Indicator definitions and interpretation
ESS1 ensemble: project average of the oil and gas import-dependency channels. Lower future values indicate lower external fossil exposure.
ESS1 oil and gas channels: oil and gas accessibility/import-exposure components.
ESS2: industrial energy intensity; lower values indicate less industrial energy per unit of GDP.
ESS3: domestic production ratio; higher values indicate stronger domestic coverage.
ESS4: directional security index. Lower ESS1 and ESS2 exposure raises the score; higher ESS3 coverage raises it. Raw paper composite is separately identified.
Renewable consumption share: higher means a larger renewable share in final consumption.
T_REN_RATE: renewable production intensity relative to system size.
T_REN_RATE_2: renewable production-to-consumption ratio.
Full formulas are centralized in the Method & Formula Atlas. Displayed source values retain their generated units and scales.
Generated scenario paths
Metric explorer
| Year | Period | Scenario | Variant | Value | Lower bound | Upper bound |
|---|
Forecast bands and variants are the generated pipeline outputs. Historical and forecast observations use the same metric source definitions as the Streamlit reference.
Source-generated chartsESS4 index, full scenario panels, forecast-quality backtest, and energy-partner analysis
This is the existing generated Plotly dashboard used by the Streamlit view. Its hover labels, legends, backtest charts, and partner-source comparisons are preserved unchanged.
Existing partner API
EU energy partners
| Rank | Partner | Share | Product | Source |
|---|
Source, product, year, unit, trade scope, and metric define each partner comparison. Current API partner observations are EU-level.
Inspect the evidence
Scenario diagnostics
Generated low / central / high variants
Sensitivity analysis
| Year | Variant | Value | Lower bound | Upper bound |
|---|
Variants use the pipeline's existing low, central, and high sensitivity scales. Baseline is not included in the legacy sensitivity selector.
Final forecast year contributions
Scenario decomposition
| Scenario | Year | Target metric | Driver | Contribution |
|---|
Contributions reproduce the generated decomposition artifact: central-variant final-year differences from Baseline; ESS1 oil and gas contributions are each halved.
Historical association, not causality
Driver relationships
| Indicator | Correlation | Standardized beta | R squared | Slope | Observations |
|---|
Historical associations are descriptive. They are not causal estimates or policy effects.
Manual registry and generated detections
Shock timing and modeled impacts
| Shock | Category / channel | Target | Start | End | Origin | Evidence | Notes |
|---|
Forecast impact overlays
| Year | Scenario | Shock | Target | Evidence | Generated impact |
|---|
Shock magnitudes are author-calibrated scenario assumptions; linked sources support context, not magnitude. Timeline items are not causal claims.
Pipeline-time screening snapshot
External geopolitical news
News matching is generated during pipeline refresh, not live monitoring. Matches are screening signals; auto-detected shocks have provisional magnitudes and are not causal estimates.
Auto-detected shocks
External-news matches
Rolling-origin evaluation
Forecast validation information
| Target year | Horizon | Actual | Predicted | Absolute percentage error | Direction | Model |
|---|
Validation checks, baseline selection, and benchmark tables
Formula and forecast checks
Selected baseline models
Historical reality benchmarks
Data lineage and recommended workflow
ESS/CODICE.py remains the historical reference for energy-dataset preparation. PIPELINE/run_all.py is the operational entry point coordinating core ESS and digital datasets, partner-source datasets, forecasts and dashboards, and validation and provenance outputs.
Energy-partner source hierarchy: Raw Comext is preferred for scholarly product-partner replication when populated; Official API is the reproducible annual quantity route; Workbook is the published comparison layer, especially for workbook-specific 2025 figures. These source layers differ in scope, units, coverage, and period.
External-news matches above reflect the most recent generated refresh snapshot. They are not live monitoring.
Refresh commands
python PIPELINE/run_all.py --refresh-eurostatFull project refresh: updates standard Eurostat inputs, rebuilds ESS and digital datasets, refreshes annual partner routes, forecasts, dashboards, validation, and provenance.
python PIPELINE/run_all.py --skip-external-newsRecommended local-development route; rebuilds analytical outputs while skipping external-news detection.
python PIPELINE/run_all.py --refresh-eurostat --refresh-comext --comext-start-year 2014 --comext-end-year 2025 --skip-external-newsPublication refresh with raw Comext partner replication.
python PIPELINE/fetch_comext_energy_imports.py --start-year 2014 --end-year 2025Lower-level Comext-only fetch for inspecting or validating its raw landing file.
python PIPELINE/auto_refresh.py --run-immediately --watch-shocks --interval-hours 24Periodic Eurostat checks and rebuilds when the local geopolitical shock registry changes.
python PIPELINE/check_grass_setup.py
.\run_project.ps1 -CheckGrassGRASS GIS wiring check, retained here because it is part of the shared application workflow reference.
Methodology and source notes
ESS1 follows the paper's import-dependency/accessibility concept; ESS2 is the industrial energy-intensity component; ESS3 is the domestic production-rate component. ESS4 is a directional, 2015-based index, while the raw composite remains separately available.
IMEC and Piano Mattei scenario magnitudes are author-calibrated exploratory assumptions. External sources provide context and mechanisms, not causal identification or calibrated effect sizes.
Backtest evaluation uses the extended Eurostat aggregate-proxy history; it is distinct from the production ESS workbook history. Inspect the source labels and horizon-specific scores before interpreting forecast performance.
Refresh status by dataset
Historical metrics and energy-partner validation
Historical metrics used in forecasts
Energy-partner validation checks
Generated partner artifacts
| Artifact | Available |
|---|
Native analytical workspace
Main Dashboard / Policy Scenarios
SecurEnergy EU energy-security indicators, policy scenarios, and country comparisons.
Loading build status
Loading indicator data...
Historical series
Trend
Latest cross-section
EU27 comparison
| Reference | Value | Reading |
|---|
The EU27 reference is the project aggregate used by the existing country indicator dataset.
Member-state comparison
EU27 ranking
| Rank | Country | Value | Distance from EU27 | Reading |
|---|
Ranking follows the selected indicator direction. It is a within-project comparison, not a universal energy-security score.
Supplier evidence
EU energy partners
| Rank | Partner | Share | Product | Source |
|---|
Partner figures are conditional on source, product, year, unit, trade scope, and metric. The current API route is EU-level, not country-level.
Indicator distinction
ESS versus conventional dependency
-
| Year | ESS1 project channel | Conventional benchmark | Source distinction |
|---|
ESS values are internally consistent project index inputs. Conventional values are percentages and should not be interpreted as ESS values.
Macro interaction
GDP and macro context
-
| Year | Value | Period | Model context |
|---|
Forecast rows are generated model outputs and should not be read as observed macroeconomic data.
Industrial production
Production index
| Year | Index | Period | Scenario/model |
|---|
Index values are generated industrial-production outputs. Forecast and scenario rows retain their pipeline phase labels.
Digital intensity
Digital adoption context
| Year | Country | DII | EU27 reference | Source |
|---|
Digital intensity is supporting context, not an ESS formula input.
Policy scenarios
Scenario paths
| Year | Scenario | Value | Interval | Variant |
|---|
Scenario paths are exploratory sensitivity exercises. Intervals describe the generated model output, not certainty about policy outcomes.
GIS member states
Spatial indicator snapshot
| Country | Value | Spatial feature |
|---|
This first native GIS slice consumes the generated country geometry and attributes. Interactive map rendering remains a separate presentation step.
Provenance
What this number means
-
- Source artifact
- -
- Data build
- -
- Publication status
- -
Interpretation note
Read direction before level
-
This workspace reports generated research outputs. It does not replace official Eurostat dependency statistics or imply causality.
SecurEnergy EU · PRODCOM
Eurostat Sold Production Concept
A EU27 view of product-level sold-production value, its nominal index, and the activity mix from official PRODCOM data.
Loading PRODCOM data
Loading generated Eurostat PRODCOM observations...
Concept and method
Current-price sold production
This companion view adds Eurostat's sold production concept without changing the project's broader all-production dashboard. It sums available product-level EUR values from PRODCOM DS-059358 for EU27_2020.
The index is a nominal rebasing of sold-production value to 2021 = 100. It is not a constant-price volume measure. Eurostat's Statistics Explained article also discusses constant-price presentation; this dashboard retains the directly reproducible PRODCOM value concept.
Current-price EUR billions
Nominal index · 2021 = 100
Year-over-year change
Backtest-selected forecast
Source, units, and scope
-
Source artifacts: -
Latest build: - · Publication status: -
-
This sold-production concept complements, rather than replaces, the project's broader industrial-production index.
EU27 aggregate
Total EU Sold Production
Current price
Sold-production value
Nominal rebasing
Sold-production index
Annual movement
Year-over-year change
Manufacturing breakdown
Activities and shares
Five highlighted activities
Nominal activity indices
Broad activity groups
Share of EU sold production
Share snapshots are shown for 2014, 2024, and the final forecast year. “Other” comprises product divisions outside the five explicitly grouped broad activities.
Rolling historical backtests
Forecast Model Selection
Forecast extensions use the candidate model that performed best in rolling historical backtests for each series.
| Series | Selected baseline | Backtest MAPE | Direction accuracy | Tests |
|---|
SecurEnergy EU · Macro interaction
Macro Interaction Dashboard
Compare nominal GDP, household electricity prices, industrial sold production, CPI, and inflation across their generated histories and backtest-selected extensions.
Loading macro data
Loading generated macro interaction observations...
How to read this view
Shared index and native units
-
The indexed chart compares each series with its own value in the first year shared by all five series. The raw-units chart keeps each series in its source unit and shows year-over-year changes; inflation-rate changes are percentage-point differences.
-
Indexed comparison
EU Macro Interaction Dashboard
Historical
Backtest-selected forecast
Index base
Pandemic / energy shock · 2020–2022
-
Raw Units Explorer
Native value and annual change
Observed value
Forecast value
Annual change
Forecast annual change
Latest observed context
Series snapshots
Rolling historical backtestsMacro Forecast Model Selection
Forecast extensions use the candidate model that performed best in rolling historical backtests for each series.
| Series | Selected baseline | Backtest MAPE | Direction accuracy | Tests |
|---|
Co-movement Snapshot
Annual-change correlation
-
Source and provenanceHow we know each series
This table documents the source and transformation used before the indexed and raw-unit charts are drawn.
| Series | Source / file | Filters / scope | How the value is known | Unit in chart | Latest observed year | Dashboard treatment |
|---|
Build: - · Publication status: -
SecurEnergy EU · Experimental research
Multi-API ESS Trend Experiment
Eurostat-based ESS and renewable-production target histories with baseline and external-feature forecast experiments.
Loading experiment artifacts
Loading generated Multi-API ESS artifacts...
Target history and scope
Exact project ESS formula targets
Eurostat remains the target formula source. The exact generated country-sum production history is shown as observed history. The gray baseline and green augmented lines are future model outputs, selected on the full available history; they are not observed values or in-sample fitted history.
Panel meanings follow the generated target labels: ESS1 oil/gas exposure ensemble; ESS2 industrial energy intensity; ESS3 domestic production ratio (DEP / TEC); renewable consumption share (% of TEC); T_REN_RATE (T_REN_PROD / TEC); and T_REN_RATE_2 (T_REN_PROD / T_REN_C). See the Method & Formula Atlas for the project’s centralized formula definitions.
-
Observed Eurostat target history
Baseline year-trend forecast
All-APIs augmented forecast
Empirical backtest interval
Six separate target panels
ESS and Renewable-Production Trends
Provenance and availability
Source Coverage and Formula Role
Coverage and publication labels below come from the generated experiment artifact. External observations supply features or context; they do not replace formula inputs.
Selected features · not causal effects
Source Features and Forecast Relationships
A source appears here when its generated variables were selected by the augmented model. The forecast shift compares the generated 2030 augmented value with the Eurostat trend baseline; it does not measure a causal source effect.
Selected source features by ESS target
2030 forecast change from baseline
Nested rolling-origin validation
Augmented Forecast Quality
Each backtest prediction uses only information available at its cutoff year. Negative MAPE or RMSE changes favor the augmented model; direction accuracy is reported as generated.
One-year MAPE · baseline vs augmented
One-year direction accuracy
Exploratory candidate grid
Source Ablation and Robustness
Each source case is shown with the best candidate model under the legacy one-year summary selection. Negative MAPE changes are lower than the Eurostat trend baseline. This candidate-grid diagnostic is separate from the nested augmented validation.
Multi-horizon robustness checks
Full-history model selection
Selected Multi-API Model
The future augmented line uses the all-APIs model family selected from the full available history. The nested backtest above is the validation-safe evidence.
Feature selection
Selected Features by ESS Target
Feature selection is performed inside each training window. The source map documents the generated lag and change variables; selection does not imply causality.
Technical provenanceVariable Source Map
Raw generated feature codes, source/resource, transformation, and meaning/unit from the legacy source map.
Supporting inputs · not target replacements
External Source Context
Comext import concentration · top partner share
Comext import concentration · HHI
World Bank/DataBank proxy indicators
Each series is indexed to its first available value from 2015. These are broad proxies, not exact Eurostat ESS inputs.
Wikipedia and GDELT shock evidence
Historical and screened event context used for interpretation; news monitoring availability is shown in the source coverage table.
OECD Energy-Related Public SDMX Dataflows
Candidate catalogue/context rows from the generated legacy dashboard.
Separate validation context
Original Project Forecast Quality
This is the original extended EU27 Eurostat aggregate-proxy backtest, separate from the exact production-history targets above. Predictions are rolling-origin results against later observed proxy history.
Backtest MAPE by metric and horizon
Direction accuracy by metric and horizon
Selected Baseline Models
Original project forecast-quality artifacts are unavailable or empty in this build.
Generated feature artifactsFeature Matrix and Model Input Previews
These are the last ten rows of the generated annual source feature matrix and its previous-year value / change inputs.
Model Inputs Preview
Source-row auditEvidence Rows Generated
Methodology and limitations
How to Interpret the Experiment
Generated artifacts:
SecurEnergy EU · Digital Intensity
Digital Intensity and ESS Scenarios
Observed EU27 digital-intensity history alongside Baseline, IMEC, and Piano Mattei scenario paths for GDP, industrial production, and ESS1–ESS3.
Loading digital-intensity data
Scenario research view
Observed history and modeled paths
This view links the generated EU27 digital-intensity indicators with GDP, industrial-production, and ESS scenario paths. The paths are scenario overlays; they do not identify causal effects or describe observed future outcomes. Historical lines end at their available observations, and the shaded zone marks the 2025–2030 scenario period.
Loading generated digital-intensity and scenario data...
Scenario overlays
Choose scenario paths to display
Historical and scenario comparisons
Digital intensity, macro paths, and ESS response
Digital adoption
Digital Intensity: All Activities
Scenario GDP
EU27 GDP Path
Digital adoption
Digital Intensity: Manufacturing
Scenario industrial output
Industrial Production: Total Industry
Energy security
ESS1 Ensemble
Energy intensity
ESS2
Domestic production
ESS3
2030 scenario snapshot
Digital All versus ESS1
HistoricalScenario periodLine color and pattern follow the generated scenario definitions.
Scenario results
2030 Summary and Scenario Timeline
For ESS1 and ESS2, lower values indicate lower exposure or intensity. For ESS3, higher values indicate stronger domestic coverage. Delta columns compare with the generated Baseline scenario.
The timeline contains generated scenario years only; historical observations are shown in the separate chart panels.
Generated Eurostat source table
Country and Sector Digital Intensity
The table retains the source dataset, indicator version, NACE scope, employment-size group, unit, EU27 reference, and generated yearly fields. The EU27 history chart uses the corresponding historical-metrics output.
Updating table filters...
Methodology and sources
How to Read These Paths
Generated artifacts:
SecurEnergy EU · Supplier evidence
Energy Partners
EU energy-import partner shares and concentration evidence across Raw Comext, the Official Eurostat API, and published workbook figures. Each source remains separate.
Loading partner data
How to read this view
Choose the source first, then check its coverage and validation status. Partner shares rank suppliers within the selected source, scope, product, metric, and year. Absolute value or quantity charts use the generated source fields; they are not interchangeable with shares. Concentration measures describe the selected supplier mix and are neither causal nor predictive.
Loading generated partner evidence...
Evidence selection
Source and Market Scope
Partner shares
Leading Import Partners
Absolute amounts
Estimated Value or Quantity
Selected market snapshot
Partner and Concentration Summary
HHI uses the legacy 0–10,000 scale. CR3 and CR5 are the combined shares of the three and five largest partners. The diversification score and risk band reproduce the legacy scorecard and do not predict future supply outcomes.
Further analysis
Rankings, Trends, and Validation
This comparison uses the selected scope, metric, and year. By-product scope compares all available products for the chosen source. It does not combine evidence across sources.
The trend uses the leading partners from the selected year, ranked under the current source and filters.
The curve orders all selected partners by share and shows cumulative share by rank.
This table shows the full selected-year ranking with the previous available source year, share change in percentage points, and dense-rank change where available.
Selected source
Source availability and reliability
Partner validation checks
Generated artifacts
Partner share is supplied by the generated source artifacts. Legacy concentration uses HHI = sum of squared decimal partner shares × 10,000. Workbook Q1 news benchmarks are retained as not-comparable diagnostics against annual 2025 workbook shares and do not count as blocking validation failures.
SecurEnergy EU · ESS input audit
Energy Table
Country-year energy indicators used to build the ESS metrics and EU27 aggregates.
Loading Energy Table
Audit and interpretation
This table is the raw analytical backbone behind the ESS formulas. Use it to audit country-year inputs and derived indicators; it is intended as a validation table rather than a presentation chart.
Some columns are component inputs and some are derived project indicators. EU27 aggregate fields retain the generated values. The four numerator/denominator fields hidden by the legacy view remain hidden here.
Loading country-year energy indicators...
Table controls
Filter the audit table
Leave Country and Year unselected to show all observations. Search is case-insensitive and supports the legacy regular-expression behavior.
Country-year inputs
Energy and ESS indicators
Page 1 of 1
Numeric values are displayed to three decimal places, matching the legacy table formatting; the API retains the source values and missing observations remain blank.
Source and methodology
Source artifact: ESS/energy_ESS_TGI_ktoe.csv. This view reuses the exact generated country-year dataset used by the legacy dashboard.
Methods and provenance
Methodology and Source Notes
The complete research design, formulas, source hierarchy, validation approach, forecast construction, interpretation guardrails, and energy-partner methodology used by the local and hosted dashboards.
Contents
Canonical Method Section
Method & Formula Atlas
This is the canonical formula section for the Streamlit app and static website. Dashboard views refer back here instead of repeating the same equation panels.
Formula notation remains selectable and copyable. The displayed expressions and interpretations follow the legacy Atlas.
ESS1 Oil Exposure
ESS1_oil = sum_i(TOI_i * TOC_i * TEC_i) / sum_i(TOC_i * TEC_i * GDP_i)
Leman Erdal; Ali Osman Pektaş; Omer Ozkan; Filiz Özkan; Security of energy supply in Japan: a key strategy and solutions; p.136-137. ESS1 is the energy import-dependency indicator. It represents the accessibility dimension of energy-supply security: supply is more secure when it can continue and be made ready for use, whether the source is domestic or foreign. This project applies that logic to the oil-import channel.
ESS1 Gas Exposure
ESS1_gas = sum_i(TGI_i * TGC_i * TEC_i) / sum_i(TGC_i * TEC_i * GDP_i)
ESS1_ensemble = (ESS1_oil + ESS1_gas) / 2
Leman Erdal; Ali Osman Pektaş; Omer Ozkan; Filiz Özkan; Security of energy supply in Japan: a key strategy and solutions; p.136-137. ESS1 is an import-dependency ratio. This project applies the same accessibility logic to gas imports. The displayed ESS1 ensemble is the average of the oil and gas channels, used as a project-level fossil-import exposure signal.
ESS2 Industrial Energy Intensity
ESS2 = sum_i(IEC_i) / sum_i(GDP_i)
Leman Erdal; Ali Osman Pektaş; Omer Ozkan; Filiz Özkan; Security of energy supply in Japan: a key strategy and solutions; p.136-137. ESS2 is energy intensity, defined as energy use relative to GDP. It captures both accessibility and affordability because lower energy intensity reduces the energy required by the economy and can lower exposure to imported-energy needs. Lower values are therefore the stronger security direction.
ESS3 Domestic Production Ratio
ESS3 = sum_i(DEP_i) / sum_i(TEC_i)
Leman Erdal; Ali Osman Pektaş; Omer Ozkan; Filiz Özkan; Security of energy supply in Japan: a key strategy and solutions; p.136-137. ESS3 is the domestic production rate of energy. It measures the share of domestic energy production in total energy consumption and belongs to the accessibility dimension. Higher domestic production coverage improves supply security; falling values imply greater fragility if external supply is interrupted.
ESS4 Directional Security Index
Displayed ESS4 is normalizedESS4_raw_formula = (ESS1_ensemble + ESS2 + ESS3) / 3
ESS4_index = mean(ESS1_ensemble_2015 / ESS1_ensemble * 100, ESS2_2015 / ESS2 * 100, ESS3 / ESS3_2015 * 100)
ESS1 and ESS2Lower is better, so the index uses the inverse ratio.
ESS3Higher is better, so the index uses the direct ratio.
Important distinction: the raw paper average is retained as ESS4_raw_formula. The dashboard ESS4 value is the 2015 = 100 directional security index, so it can be much higher than the raw average.
ESS4 is a direction-normalized security index with 2015 = 100, so read it as a trajectory where higher values mean stronger composite security.
Renewable Consumption Share
renewables_share_t = 100 * T_REN_C_t / TEC_t
Higher values indicate a larger renewable share in final consumption.
Renewable Production Rates
T_REN_RATE = sum_i(T_REN_PROD_i) / sum_i(TEC_i)
T_REN_RATE_2 = sum_i(T_REN_PROD_i) / sum_i(T_REN_C_i)
Renewable production relative to total energy demand and renewable consumption.
Renewable Production Intensity (T_REN_RATE): Higher values indicate stronger renewable production relative to system size.
Renewable Production-to-Consumption Ratio (T_REN_RATE_2): Higher values indicate domestic renewable production is keeping up better with renewable use.
Conventional Dependency Benchmarks
OilDependency_conv = sum_i(TOI_i) / sum_i(TOC_i)
GasDependency_conv = sum_i(TGI_i) / sum_i(TGC_i)
Direct import-to-consumption ratios used only as a benchmark against ESS channels.
Comext Partner Shares
share_pct = 100 * partner_value / total_product_value
HHI = sum_j((share_pct_j / 100)^2) * 10000
Raw Comext product-partner evidence for import value, concentration, and supplier exposure. Higher HHI values indicate more concentration.
Forecast Validation
MAPE = mean(abs(actual - forecast) / abs(actual))
direction_accuracy = mean(sign(delta_actual) = sign(delta_forecast))
Rolling-origin backtests evaluate whether forecast paths reproduce direction and scale before scenario overlays.
Methodology and Source Notes
The complete research design, formulas, source hierarchy, validation approach, forecast construction, interpretation guardrails, and energy-partner methodology used by the local and hosted dashboards.
Open Methodology & SourcesBuild quality and disclosure
Publication Readiness Issues
Live publication-readiness register showing the current generated checks, open warnings or blockers, evidence files, resolution paths, and disclosure language for each tracked issue family.
Loading publication readiness
Loading generated publication-readiness checks...
External news screening was skipped in this rebuild. That does not lower the publication-readiness status, but this artifact should not be described as refreshing live news-monitoring evidence.
Generated artifact
Generated Check Table
No readiness table found at DATA/generated/publication_readiness_checks.csv.
| Category | Issue | Status | Blocking | Generated Detail |
|---|
Causes and actions
Detailed Issue Register
No generated issue rows are available to expand.
Register Document
Durable Markdown register: docs/PUBLICATION_READINESS_ISSUES.md
Open full Markdown register
Publication Readiness Issue Register
Last reviewed after the live publication rebuild completed on 2026-07-03 at 12:08:33 UTC.
This register is the durable audit trail for publication-readiness issues. The live status is generated by the pipeline in DATA/generated/publication_readiness_checks.csv and summarized in DATA/generated/refresh_metadata.json; this document explains what every issue family means, why it matters, what evidence to inspect, how to resolve it, and how to disclose it if it remains open in a publication-facing build.
Current Status
The latest generated status is ready.
Current evidence:
DATA/generated/publication_readiness_checks.csvreportspassfor every tracked row.DATA/generated/refresh_metadata.jsonreports"publication_readiness": {"status": "ready", "blockers": [], "warnings": []}.- 10 critical Eurostat input files were refreshed through the live SDMX route.
- The official Eurostat API partner dataset was rebuilt from live responses and did not use the cached fallback.
- The raw Comext replication layer was refreshed for 2014-2025 and rebuilt successfully.
- Core formula and forecast validation passed 10 of 10 checks.
Resolved Issues From The Previous Build
Full Eurostat Refresh Was Not Requested
Previous state: warn.
What happened: the earlier build was run without --refresh-eurostat. That does not mean the formulas were wrong, but it does mean the build could not claim that the critical Eurostat inputs were freshly downloaded during that rebuild. The pipeline therefore marked the project as ready_with_limitations rather than ready.
Resolution: the documented publication refresh command was run with live network access:
.\.venv\Scripts\python PIPELINE\run_all.py --refresh-eurostat --refresh-comext --comext-start-year 2014 --comext-end-year 2025 --skip-external-newsResult: every critical input listed in the refresh status was refreshed by SDMX, with no critical local-file reuse.
Official API Partner Source Reused Cached Output
Previous state: warn.
What happened: the official Eurostat annual partner API request previously failed with a connection reset, so the pipeline reused the last successful generated DATA/generated/eu_energy_partner_official_api_annual.csv. That fallback preserved reproducibility, but it also meant the partner-source layer was not live-refreshed during that run.
Resolution: the full rebuild fetched the official partner API data successfully. The latest metadata reports "official_api_used_cache_fallback": false and an empty fallback reason.
Result: the official API issue is closed for the latest build.
Status Semantics
The dashboard label is intentionally conservative:
readymeans every publication-readiness check passed.ready_with_limitationsmeans no blocking check failed, but at least one warning or non-blocking failure exists. The project can still be shown, but the limitation should be disclosed.blockedmeans at least one blocking failure exists. A blocked build should not be treated as publication-ready until the blocking issue is fixed and the pipeline is rerun.
The rule is implemented in PIPELINE/STREAMLIT.py for display and in PIPELINE/pipeline_core.py for generated metadata.
Issue Family: Critical Eurostat Inputs
Tracked row: core_data_refresh / critical_eurostat_inputs.
Why this issue exists: the ESS formulas depend on a defined set of critical Eurostat-derived workbooks: GDP, total energy consumption, industrial energy consumption, oil consumption, gas consumption, renewable consumption, renewable production, oil imports, gas imports, and domestic energy production. If a publication build claims freshness, those inputs must be refreshed from Eurostat rather than silently reused from old local files.
How it becomes a warning: the rebuild is run without --refresh-eurostat. In that case, the pipeline may still be analytically coherent because it uses the local input files already present in the repository, but it cannot honestly claim that the build refreshed the critical Eurostat backbone.
How it becomes blocking: the rebuild is run with --refresh-eurostat, but one or more critical inputs cannot be fetched through SDMX or legacy fallback and would have to reuse local files. That is worse than an intentional no-refresh build, because the operator explicitly requested a fresh publication rebuild and the source-refresh promise was not met.
Why it matters: the ESS trends and forecast history are only as current as the input tables. A stale GDP, energy consumption, import, or production workbook can move the denominator or numerator of ESS1, ESS2, ESS3, renewable rates, and validation targets. The risk is not merely cosmetic; a late Eurostat revision can change historical levels, annual turning points, and fitted forecast baselines.
Evidence to inspect:
DATA/generated/refresh_status.csvDATA/generated/refresh_metadata.json- the root-level critical workbooks such as
GDP.xlsx,TEC.xlsx,IEC.xlsx,TOC.xlsx,TGC.xlsx,T_REN_C.xlsx,T_REN_PROD.xlsx,TOI.xlsx,TGI.xlsx, andDEP.xlsx
Resolution path:
- Run the publication refresh with
--refresh-eurostat. - Confirm
DATA/generated/refresh_status.csvreportssdmx_refreshedorlegacy_refreshedfor every critical file. - Confirm
datasets_reused_local_countis 0 for critical inputs. - Rerun the dashboards and validations as part of
PIPELINE/run_all.py.
Disclosure if still open: say that the displayed analysis is based on the local Eurostat input snapshot bundled with the project and should not be described as a freshly refreshed Eurostat build.
Issue Family: Core Formula And Forecast Validation
Tracked row: core_validation / formula_and_forecast_checks.
Why this issue exists: the project publishes derived indicators, not raw tables alone. The validation layer checks that the generated historical metrics match the formulas and that forecast intervals and central estimates obey basic consistency rules.
How it fails: one or more rows in DATA/generated/validation_checks.csv has status fail.
Why it matters: a formula validation failure means at least one displayed metric no longer matches the documented formula, source transformation, or forecast sanity rule. That is a publication blocker because the dashboard would be asking readers to trust a number whose own consistency audit failed.
Evidence to inspect:
DATA/generated/validation_checks.csvDATA/generated/historical_metrics.csvDATA/generated/policy_scenario_forecasts.csvPIPELINE/scenario_dashboard.pyPIPELINE/pipeline_core.py
Resolution path:
- Open the failing validation row and identify the metric, value, threshold, and detail.
- Trace the metric back to the source table and formula implementation.
- Fix the formula implementation, source transformation, missing-data handling, or validation threshold only if the threshold is demonstrably wrong.
- Rerun
PIPELINE/run_all.py. - Confirm every validation row passes.
Disclosure if still open: do not publish the affected metric as validated. If the build must be shared internally, mark the exact metric as under formula-validation review and exclude it from claims.
Issue Family: Workbook Partner Validation
Tracked row: energy_partners / workbook_partner_validation.
Why this issue exists: the workbook partner-share layer is useful because it reflects published workbook-style partner evidence, including workbook-specific 2025 figures. It is also more fragile than the raw Comext route because workbook supplements can encode partial-year scope, product definitions, partner labels, or formatting conventions that are not always directly comparable to the raw official trade extract.
How it fails: blocking partner validation checks fail in DATA/generated/energy_partner_validation.csv.
Why it matters: partner-share claims are easy to overstate. A workbook layer might be directionally useful but not clean enough for publication-default charts if it fails blocking validation. The pipeline can quarantine the workbook route while retaining the raw Comext and official API routes.
Evidence to inspect:
DATA/generated/energy_partner_validation.csvDATA/generated/eu_energy_partner_shares.csvDATA/generated/eu_energy_partner_shares_from_comext.csvDATA/generated/eu_energy_partner_official_api_annual.csvDATA/energy_partner_methodology.md
Resolution path:
- Read the failing validation rows and separate blocking failures from
not_comparablediagnostics. - Check whether the workbook issue is a parsing error, a source-scope mismatch, a product mapping mismatch, a partner-code mismatch, or an expected non-comparability.
- Fix parsing or mapping issues in the workbook builder.
- Keep legitimate scope differences documented rather than forcing artificial agreement.
- Rerun the partner build and validation.
Disclosure if still open: use raw Comext as the publication-default partner source and describe workbook figures as a supplementary comparison layer, not as the primary replicated evidence.
Issue Family: Official API Live Fetch
Tracked row: energy_partners / official_api_live_fetch.
Why this issue exists: the official Eurostat annual partner API route is the clean reproducible annual quantity route for partner analysis. The pipeline allows a cached fallback so a transient API outage does not destroy local reproducibility, but a fallback is still a limitation for a freshness claim.
How it becomes a warning: PIPELINE/fetch_energy_partner_official_api.py catches a request failure or receives no requested annual slices and returns the existing generated CSV with used_cache_fallback = True.
Why it matters: cached fallback means the chart can still be internally consistent, but it was not refreshed from the live official API in that run. If Eurostat revised the annual partner series after the cached file was generated, the displayed annual quantities may lag the source.
Evidence to inspect:
DATA/generated/refresh_metadata.json, especiallyofficial_api_used_cache_fallbackandofficial_api_fallback_reasonDATA/generated/eu_energy_partner_official_api_annual.csv- console logs from
PIPELINE/run_all.py
Resolution path:
- Rerun the full pipeline when Eurostat API connectivity is stable.
- Confirm the metadata reports
"official_api_used_cache_fallback": false. - Confirm official API partner rows are nonzero.
- If the API repeatedly fails, inspect the request URL, Eurostat API availability, proxy settings, and local network permissions.
Disclosure if still open: say that the official API partner layer reused the last successful generated output after a live request failure. Prefer the raw Comext replication layer for publication-grade product-partner claims if it is populated.
Issue Family: Raw Comext Replication
Tracked row: energy_partners / raw_comext_replication.
Why this issue exists: raw Comext is the strongest partner-share evidence route when the claim is about product-partner import evidence. It starts from the filtered trade records instead of a published workbook summary.
How it becomes a warning: DATA/generated/eu_energy_partner_shares_from_comext.csv is missing. The project can still use the official API annual quantity route, but it loses the preferred raw replication layer.
Why it matters: without raw Comext, the project has less direct control over product scope, trade-value and net-mass evidence, partner aggregation, and concentration calculations. That weakens replication language around supplier exposure.
Evidence to inspect:
DATA/raw/comext_energy_imports.csvDATA/generated/eu_energy_partner_shares_from_comext.csvPIPELINE/fetch_comext_energy_imports.pyPIPELINE/build_energy_partner_from_comext_raw.py
Resolution path:
- Run the pipeline with
--refresh-comext. - Confirm the raw landing file has rows for the requested product codes and years.
- Confirm the generated raw Comext share table is populated.
- Check that product labels, partner labels, trade values, and net mass are present before using the charts.
Disclosure if still open: do not call partner charts raw-Comext-replicated. Use the official API annual route and describe the missing raw Comext layer as a pending replication enhancement.
Issue Family: Placeholder Or Unverified Shock Exclusion
Tracked row: geopolitical_shocks / placeholder_shock_exclusion.
Why this issue exists: scenario forecasts can include geopolitical shocks, but only source-noted shocks should affect publication-facing overlays. Placeholder or unverified manual events can be useful while designing scenarios, but they should not silently enter the final forecast layer.
How it becomes a warning: the pipeline excludes one or more placeholder or unverified manual shocks and records them in DATA/generated/excluded_shocks.csv.
Why it matters: a shock overlay can move forecast paths and interpretation. If the project quietly excluded shocks, the scenario count and active shock count may be lower than a user expected from the raw manual file. That is not necessarily a defect; it is a transparency issue that should be visible.
Evidence to inspect:
DATA/geopolitical_shocks.csvDATA/generated/excluded_shocks.csvDATA/shock_methodology.mdPIPELINE/scenario_dashboard.py
Resolution path:
- Inspect excluded shocks.
- Add source evidence, dates, scope, and persistence assumptions for legitimate shocks.
- Remove placeholders that should never be used.
- Rerun the pipeline and confirm the exclusion count is 0, or document why exclusions remain.
Disclosure if still open: state that some manual shock candidates were excluded from the forecast overlay because they were placeholders or lacked sufficient source notes.
Issue Family: Forecast-Relevant Source-Noted Shocks
Tracked row: geopolitical_shocks / forecast_relevant_source_noted_shocks.
Why this issue exists: the policy-scenario layer is more informative when at least one source-noted shock is relevant to the forecast window. A build with no forecast-relevant shocks can still produce baseline scenario paths, but the shock overlay becomes empty.
How it becomes a warning: no source-noted shock overlaps the forecast period after accounting for start year, end year, and persistence years.
Why it matters: if the dashboard narrative discusses shocks, but the generated forecast contains no source-noted forecast-relevant shock, the narrative and data are out of sync. This is a non-blocking limitation because some analyses intentionally run without shocks.
Evidence to inspect:
DATA/geopolitical_shocks.csvDATA/generated/excluded_shocks.csvDATA/generated/policy_scenario_forecasts.csvDATA/shock_methodology.md
Resolution path:
- Confirm whether the intended publication claim requires shock overlays.
- If yes, add or repair source-noted shock rows with valid timing and persistence.
- If no, keep the build but describe it as a scenario forecast without active shock overlays.
Disclosure if still open: say that scenario forecasts run without source-noted forecast-relevant shock overlays.
Issue Family: Digital Observation History
Tracked row: digital_series / observation_history.
Why this issue exists: the digital-intensity scenario layer uses annual observations to link digital adoption with GDP, industrial production, and ESS trajectories. Too few years of digital observations can make trend interpretation brittle.
How it becomes a warning: the digital dataset has fewer than 5 distinct observation years after generation.
Why it matters: with a short digital history, fitted relationships can be dominated by one or two revisions, missing years, or structural breaks. The digital dashboard can still be useful, but claims about stable historical relationships become weaker.
Evidence to inspect:
TECH_IND/digital_intensity_ml.csvDATA/generated/refresh_metadata.json, especiallydigital_observation_year_countPIPELINE/pipeline_core.py
Resolution path:
- Rebuild the digital dataset from official sources.
- Confirm the generated table has at least 5 distinct years.
- If the official source genuinely has limited history, keep the warning and limit claims to exploratory interpretation.
Disclosure if still open: describe digital-intensity relationships as short-history exploratory evidence rather than stable long-run estimates.
Operational Caveat: External News Screening
Tracked in metadata, not currently a publication-readiness warning.
The latest ready build was run with --skip-external-news. That is acceptable for a local publication rebuild when the project is not making live news-monitoring claims. It does mean the build should not be described as having refreshed the external news screening layer during this run.
Evidence to inspect:
DATA/generated/refresh_metadata.json, especiallyexternal_news_skippedDATA/generated/external_news_matches.csvPIPELINE/news_detector.py
Disclosure language: "The publication build refreshed Eurostat, official API, and Comext layers. External news screening was skipped for this local rebuild, so no live news-monitoring claim is made for this artifact."