Build an energy market research agent

Energy market intelligence — electricity prices, generation capacity, grid congestion, renewable project pipelines, commodity curves, regulatory rate cases — is some of the most expensive data in the economy. Bloomberg Terminal, Wood Mackenzie, and S&P Global Commodity Insights charge five and six figures a year for it. But most of the underlying signal is public: the EIA publishes generation and price data, FERC dockets are open, every ISO/RTO posts load and capacity data, and energy news moves in real time. This guide builds a Python agent that mines those public sources and assembles a structured energy market brief. It chains all four Superhighway endpoints — /research for the structure of an energy market, /search for EIA reports, FERC filings, and ISO/RTO data, /scrape for price tables and capacity figures, and /news for price spikes, outages, project announcements, and regulatory decisions — then uses an LLM to emit a structured brief as JSON.

Energy prices and grid conditions change in real time. This agent synthesizes publicly available reports and news. Always verify current prices and capacity figures with real-time sources — EIA.gov, your ISO/RTO's website, and commodity exchanges — before making any trading or investment decision.

1. What you'll build

A Python agent that takes an energy market topic, commodity, region, or technology and produces a structured energy market intelligence brief:

• Synthesizes the market: structure, pricing dynamics, capacity mix, demand trends, regulatory framework
• Finds EIA reports, ISO/RTO data, FERC filings, and commodity price surveys
• Scrapes energy agency pages for specific price tables and capacity data
• Pulls recent energy news: price moves, outages, project announcements, policy decisions, weather impacts
• Uses an LLM to generate a structured brief — price environment, capacity outlook, demand trends, market drivers, regulatory landscape, and supply risks as JSON

2. Setup

pip install openai requests python-dotenv

Create a .env file with your two keys:

SUPERHIGHWAY_API_KEY=your_key_here
OPENAI_API_KEY=your_key_here

3. Research the energy market landscape

Start with /research, which pulls multi-source background into one synthesis — how the market is structured, what's driving prices, the capacity mix and its trajectory, demand-side trends, and the regulatory environment. This is the grounding context for every later step.

import requests, os, json

SUPERHIGHWAY_KEY = os.getenv("SUPERHIGHWAY_API_KEY")
BASE = "https://superhighway.walls.sh"

def research_market(energy_topic: str) -> str:
    """Deep synthesis: market structure, pricing, capacity, demand, regulation."""
    r = requests.get(
        f"{BASE}/research",
        params={
            "q": f"{energy_topic} market price capacity demand supply regulatory outlook",
            "pages": 6
        },
        headers={"Authorization": f"Bearer {SUPERHIGHWAY_KEY}"}
    )
    data = r.json()
    return data.get("synthesis", data.get("markdown", ""))[:3000]

4. Find market data and reports

Two /search calls do the legwork. The first hunts for official data sources — EIA reports, FERC filings, ISO/RTO data, generation and price figures; the second goes after analysis and outlook: market commentary, investment notes, renewable capacity additions, and grid studies. Together they give the agent both the hard numbers and the interpretive context.

def find_market_data(energy_topic: str) -> list[dict]:
    """Find official data sources: EIA, FERC, ISO/RTO, generation and price data."""
    r = requests.get(
        f"{BASE}/search",
        params={
            "q": f"{energy_topic} EIA report price capacity generation FERC ISO RTO data 2024 2025",
            "limit": 6
        },
        headers={"Authorization": f"Bearer {SUPERHIGHWAY_KEY}"}
    )
    return r.json().get("results", [])

def find_analysis(energy_topic: str) -> list[dict]:
    """Find analysis and outlook reports: market commentary, capacity, investment."""
    r = requests.get(
        f"{BASE}/search",
        params={
            "q": f"{energy_topic} market analysis outlook investment renewable capacity addition grid",
            "limit": 6
        },
        headers={"Authorization": f"Bearer {SUPERHIGHWAY_KEY}"}
    )
    return r.json().get("results", [])

5. Scrape energy data pages

/scrape turns each candidate URL into clean, LLM-ready markdown. This is where the agent pulls actual price tables, capacity figures, generation mix, and regulatory text out of EIA pages, ISO/RTO dashboards, FERC filings, and energy publications.

def scrape_data_page(url: str) -> dict:
    """Scrape an energy data page for price tables, capacity figures, and regulation."""
    r = requests.get(
        f"{BASE}/scrape",
        params={"url": url},
        headers={"Authorization": f"Bearer {SUPERHIGHWAY_KEY}"}
    )
    data = r.json()
    return {
        "url": url,
        "title": data.get("title", ""),
        "content": data.get("markdown", "")[:2500],
    }

6. Get recent energy market news

/news surfaces what just happened — price spikes, plant and transmission outages, new project announcements, regulatory decisions, extreme weather impacts, and capacity retirements. This is the time-sensitive layer that a static report scan misses, and in energy markets it's often the layer that moves prices.

def get_market_news(energy_topic: str) -> list[dict]:
    """Get recent price moves, outages, project announcements, and policy decisions."""
    r = requests.get(
        f"{BASE}/news",
        params={
            "q": f"{energy_topic} energy market price outage capacity renewable grid policy",
            "count": 6
        },
        headers={"Authorization": f"Bearer {SUPERHIGHWAY_KEY}"}
    )
    return r.json().get("articles", [])

7. Generate the energy brief with an LLM

Now hand everything to the LLM. The system prompt forbids inventing prices, capacity figures, or project dates and bars any trading recommendation — the agent summarizes what's in the sources and flags that figures need real-time verification. The output is structured JSON so it slots straight into a morning market brief, an IRP memo, or a project screening dashboard.

from openai import OpenAI

llm = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

FRESHNESS_NOTE = (
    "Energy market data moves quickly. Verify prices and capacity figures with "
    "real-time sources (EIA, ISO/RTO, commodity exchanges) before making trading "
    "or investment decisions."
)

def generate_brief(
    energy_topic: str,
    market: str,
    data_pages: list[dict],
    analysis: list[dict],
    news: list[dict],
    context: dict
) -> dict | None:
    """Generate a structured energy market intelligence brief."""

    data_text = "\n".join(
        f"- {p['title']}: {p['content'][:400]}"
        for p in data_pages[:5]
        if p.get("content")
    )

    analysis_text = "\n".join(
        f"- {r.get('title', '')}: {r.get('description', '')}"
        for r in analysis[:5]
    )

    news_text = "\n".join(
        f"- {n.get('title', '')} ({n.get('source', '')})"
        for n in news[:6]
    )

    segment = context.get("market_segment", "mixed")
    region = context.get("region", "global")
    focus = context.get("focus", "all")

    response = llm.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": f"""You are an energy market analyst preparing a research brief.
The focus is a {focus} view of the {segment} segment in the {region} market.
Be specific and factual. Only use information from the provided sources.
Do not invent prices, capacity figures, project dates, or regulatory outcomes — if a
detail isn't in the sources, say 'not found in sources.' Do not make trading
recommendations; you summarize public information, you do not advise. Always note that
prices and capacity figures must be verified against real-time sources."""
            },
            {
                "role": "user",
                "content": f"""Write an energy market brief for: {energy_topic}

Market Synthesis:
{context.get('landscape', '')[:2000]}

Data Pages (scraped):
{data_text}

Analysis and Outlook:
{analysis_text}

Recent Energy News:
{news_text}

Return JSON with:
- energy_topic: string (the topic, commodity, region, or technology researched)
- market_segment: "electricity" | "natural-gas" | "oil" | "renewables" | "hydrogen" | "energy-storage" | "mixed"
- geographic_scope: string (which markets/regions this covers)
- price_environment: string (current pricing conditions, recent moves, outlook)
- capacity_outlook: string (generation/production capacity trends, additions, retirements)
- demand_trends: string (load growth, electrification trends, demand-side factors)
- key_market_drivers: list of 3-4 strings (what's moving prices and capacity decisions right now)
- regulatory_landscape: string (key regulations, rate cases, policy incentives affecting this market)
- supply_risks: list of 2-3 strings (weather, fuel supply, geopolitical, infrastructure risks)
- recent_developments: list of 3 strings (from news — price events, project announcements, policy decisions)
- investment_signals: string (what this data suggests for investment/planning decisions, based on public data only)"""
            }
        ],
        response_format={"type": "json_object"}
    )

    try:
        brief = json.loads(response.choices[0].message.content)
        brief["data_freshness_note"] = FRESHNESS_NOTE
        return brief
    except (json.JSONDecodeError, KeyError):
        return None

8. The full research pipeline

Wire the steps together: research the market, find data and analysis, scrape the top data pages, pull news, then generate the brief. The context dict tells the agent which segment to slant toward, which region to cover, and whether you care most about prices, capacity, or policy.

def research_energy_market(
    topic: str,
    context: dict | None = None,
    max_pages: int = 5
) -> dict | None:
    """
    Run the full energy market research pipeline.

    context: {
        "market_segment": "electricity" | "natural-gas" | "oil" | "renewables" | "hydrogen" | "storage",
        "region": "US" | "EU" | "APAC" | "global",
        "focus": "prices" | "capacity" | "policy" | "all"
    }
    """
    if context is None:
        context = {"market_segment": "mixed", "region": "global", "focus": "all"}

    print(f"Researching energy market: {topic}")

    # Step 1: Market synthesis
    print("Synthesizing market landscape...")
    landscape = research_market(topic)
    context["landscape"] = landscape

    # Step 2: Find data and analysis
    print("Finding market data and analysis...")
    market_data = find_market_data(topic)
    analysis = find_analysis(topic)

    # Step 3: Scrape the top data pages
    print(f"Scraping up to {max_pages} data pages...")
    data_pages = []
    seen = set()
    for result in market_data:
        url = result.get("url")
        if not url or url in seen:
            continue
        seen.add(url)
        page = scrape_data_page(url)
        if page["content"]:
            data_pages.append(page)
        if len(data_pages) >= max_pages:
            break

    # Step 4: Recent energy news
    print("Pulling recent energy news...")
    news = get_market_news(topic)

    # Step 5: Generate the brief
    print("Generating energy market brief...")
    return generate_brief(
        topic, context.get("market_segment", "mixed"), data_pages, analysis, news, context
    )

def print_brief(brief: dict):
    if not brief:
        print("Could not generate brief.")
        return

    print(f"\n{'='*60}")
    print(f"Energy Market Brief: {brief.get('energy_topic', 'Topic')}")
    print(f"Segment: {brief.get('market_segment', '?')}  |  Scope: {brief.get('geographic_scope', '?')}")
    print(f"{'='*60}")

    print(f"\nPrice Environment:\n{brief.get('price_environment', '')}")
    print(f"\nCapacity Outlook:\n{brief.get('capacity_outlook', '')}")
    print(f"\nDemand Trends:\n{brief.get('demand_trends', '')}")

    drivers = brief.get("key_market_drivers", [])
    if drivers:
        print("\nKey Market Drivers:")
        for d in drivers:
            print(f"  * {d}")

    print(f"\nRegulatory Landscape:\n{brief.get('regulatory_landscape', '')}")

    risks = brief.get("supply_risks", [])
    if risks:
        print("\nSupply Risks:")
        for risk in risks:
            print(f"  ! {risk}")

    developments = brief.get("recent_developments", [])
    if developments:
        print("\nRecent Developments:")
        for dev in developments:
            print(f"  # {dev}")

    print(f"\nInvestment Signals:\n{brief.get('investment_signals', '')}")
    print(f"\n{brief.get('data_freshness_note', '')}")

if __name__ == "__main__":
    import sys

    # Usage: python agent.py "ERCOT Texas electricity market" electricity
    query = sys.argv[1] if len(sys.argv) > 1 else "ERCOT Texas electricity market"
    segment = sys.argv[2] if len(sys.argv) > 2 else "electricity"

    CONTEXT = {
        "market_segment": segment,
        "region": "US",
        "focus": "all",
    }

    brief = research_energy_market(query, CONTEXT, max_pages=5)
    if brief:
        print_brief(brief)

9. Common research topics

• Electricity markets — "ERCOT Texas electricity market", "PJM capacity market 2025", "California grid reliability duck curve".
• Renewables — "US offshore wind capacity additions 2025", "utility-scale solar project development pipeline", "battery storage market growth".
• Natural gas — "Henry Hub natural gas price drivers", "LNG export capacity US 2025", "European natural gas supply security".
• Regulatory — "IRA tax credits clean energy investment", "FERC transmission planning reform", "state renewable portfolio standards".
• Emerging technologies — "green hydrogen electrolyzer cost trends", "long-duration energy storage deployment", "nuclear SMR project pipeline".

10. Use cases

• Energy analyst — morning market brief: what moved overnight and why?
• Renewable energy developer — capacity and regulatory landscape before scoping a new project.
• Utility planner — demand and capacity outlook for integrated resource planning.
• Clean energy investor — market conditions and policy landscape before committing capital.
• Energy trader — supply/demand fundamentals brief before a trading session.
• Policy researcher — regulatory landscape and market structure analysis.

11. Extending the agent

• Daily market brief — schedule the agent daily with /news?q={market}+energy+price+grid for a morning energy digest.
• Project screening — run for target markets before a renewable energy project development decision.
• Policy tracker — monitor FERC dockets and IRA implementation with /news?q=FERC+ruling+clean+energy+policy.
• Commodity monitor — track Henry Hub, Brent, and TTF weekly with a structured trend comparison.

12. Data freshness

Energy prices and grid conditions change in real time. This agent synthesizes publicly available reports and news — always verify current prices and capacity figures with real-time sources (EIA.gov, your ISO/RTO's website, commodity exchanges) before making trading or investment decisions. The agent is built to give you a fast, structured starting point, not a substitute for live market data.

13. Getting your API key

Grab a free Superhighway key at /pricing (1,000 calls/month, no credit card). For an agent that provisions its own access, skip the key entirely with x402: it pays $0.002 per call in USDC on Base — no signup, no key management. See the x402 pay-per-call guide for the wallet setup.

For related builds, the ESG research agent applies the same four-endpoint pattern to corporate sustainability profiles, and the supply chain research agent uses it on fuel and infrastructure supply risk.