Quant Trading for Programmers 05: Data Matters More Than Strategy

The first four articles established boundaries, the engineering skeleton, core concepts, and A-share trading rules.

Now many people may ask: can we finally write strategies? Not yet. In quant systems, the most underestimated part is usually not strategy, but data. Dirty data, missing data, and data with the wrong policy can make any strategy look useful and make any backtest meaningless.

This article enters the data layer.

Before Strategy, Ask Where Data Comes From

A simple moving-average strategy appears to need only close prices.

But once it becomes a system, questions multiply immediately:

• Where does the stock list come from?
• How are delisted stocks handled?
• Do suspended days have K-lines?
• Is volume measured in shares, lots, or a vendor-specific unit?
• Are daily bars forward-adjusted, backward-adjusted, or unadjusted?
• Does financial-report data respect disclosure latency?
• Should there be fallback when a data source fails?
• Can fallback data enter formal backtests?

If these questions are not solved in the data layer, the strategy layer will be full of patches.

Data Sources Must Not Be Hardcoded In Strategies

Strategies should consume normalized data. They should not know vendor details.

If a strategy needs daily bars, it should not build QVeris, Eastmoney, Tushare, or iFinD request parameters by itself. It should call one unified interface and receive standardized MarketBar objects.

ZiQuant uses DataSourceConfig to manage data-source configuration:

class DataSourceConfig(Base):
    __tablename__ = "zi_quant_data_source_configs"

    name = mapped_column(String(80), unique=True)
    adapter = mapped_column(String(40), index=True)
    enabled = mapped_column(Boolean, default=True)
    priority = mapped_column(Integer, default=100)
    config_json = mapped_column(JSONB, default=dict)
    secret_ref = mapped_column(String(200), nullable=True)
    last_status = mapped_column(JSONB, default=dict)

Several fields are important.

adapter identifies the vendor type, such as qveris, eastmoney, or tushare. priority controls source priority. secret_ref references environment variables or a secret system instead of storing secrets directly. last_status records the most recent call state for troubleshooting.

With this design, strategies do not care who provides the data. Vendors can change while strategy logic stays put.

Market Data Needs Provenance

A market-bar table is not only prices.

ZiQuant’s MarketBar includes source and payload:

class MarketBar(Base):
    __tablename__ = "zi_quant_market_bars"

    symbol = mapped_column(String(16), index=True)
    trade_date = mapped_column(Date, index=True)
    frequency = mapped_column(String(16), default="1d", index=True)
    open = mapped_column(Float)
    high = mapped_column(Float)
    low = mapped_column(Float)
    close = mapped_column(Float)
    volume = mapped_column(Float, default=0)
    amount = mapped_column(Float, default=0)
    source = mapped_column(String(40), default="unknown", index=True)
    payload = mapped_column(JSONB, default=dict)

If one row comes from Eastmoney, another from QVeris, and another from fallback, the system must distinguish them. Otherwise, when a backtest looks abnormal, it is hard to know whether the strategy is wrong or the data policy is mixed.

Data provenance is not audit obsession. It is the entry point for debugging.

Financial Reports Have Latency

Price data usually updates by trading day. Financial reports do not.

A financial report has a report period and a disclosure time. A 2025 annual report is not automatically visible on the last trading day of 2025. If a backtest uses a report before it was disclosed, it introduces look-ahead bias.

ZiQuant’s current FinancialReport stores basic fields first:

class FinancialReport(Base):
    __tablename__ = "zi_quant_financial_reports"

    symbol = mapped_column(String(16), index=True)
    report_date = mapped_column(Date, index=True)
    report_type = mapped_column(String(40), default="income")
    revenue = mapped_column(Float, nullable=True)
    net_profit = mapped_column(Float, nullable=True)
    roe = mapped_column(Float, nullable=True)
    source = mapped_column(String(40), default="unknown", index=True)
    payload = mapped_column(JSONB, default=dict)

Later we will add disclosure time and policy fields. The principle is already fixed: a financial factor cannot treat “report period” as “information visible on a tradable day”.

Fallback Data Is Emergency Support, Not A Disguise

In early development, fallback data can help pages and interfaces run. That is fine, but fallback must be clearly marked.

ZiQuant’s README already states that real market and financial data will mark real sources, while fallback data will be explicitly marked as simulated_fallback. Production quality checks will also count fallback ratio.

This rule is important. Fallback is suitable for local development, UI integration, and empty-path testing. It must not pretend to be real market data in formal strategy conclusions.

Data Quality Should Become An API

Data quality should not depend on a person opening the database and looking around.

ZiQuant already prepares several endpoints:

GET  /api/data/quality
GET  /api/data/provenance
POST /api/data/coverage
POST /api/stocks/sync
POST /api/data/sync
POST /api/financials/sync
POST /api/financials/coverage

These endpoints answer different questions:

• quality: overall quality, coverage, freshness, and fallback ratio.
• provenance: data sources, row counts by source, and recent sync status.
• coverage: whether a stock batch has enough market data.
• sync endpoints: trigger stock, market, and financial-data sync jobs.

Before writing strategies, we will use these endpoints to decide whether the data can support backtesting.

Define The Data-Source Interface First

Article 5 does not implement a full vendor SDK, but the interface boundary should be clear.

A data-source adapter should provide at least three capabilities:

class MarketDataProvider:
    async def get_stock_basic(self) -> list[dict]:
        ...

    async def get_daily_bars(self, symbol: str, start: str, end: str) -> list[dict]:
        ...

    async def get_financial_reports(self, symbol: str, limit: int = 8) -> list[dict]:
        ...

The returned values are not final ORM objects yet. They are structures after the first layer of vendor parsing. Before writing to the database, they still need normalization, deduplication, validation, and source marking.

Minimum Acceptance For The Data Layer

The data layer is not done after writing one download script. It should at least answer:

• How many stock master records exist?
• How many stocks are in the public universe?
• How many daily bars does each stock have?
• What is the latest bar date?
• How much data comes from real vendors?
• What is the fallback ratio?
• How many stocks have financial-report coverage?
• What was the latest error when sync failed?

ZiQuant’s /ready, /api/data/quality, and /api/data/provenance will gradually take on these checks.

Runnable Foundation Check

Article 5 emphasizes data-quality boundaries, so the foundation check intentionally includes one invalid OHLC row. The current project uses this command to reproduce the foundation capabilities from articles 01-08:

uv run python -m scripts.chapter_examples foundation-check

The output for this chapter is:

raw_rows=3, clean_rows=2, and rejected_rows=1 show that the system did not silently swallow abnormal market data. The first stage does not chase many data sources. It first gives bad data explicit rejection reasons.

Hands-On Task

This article first confirms the existing data models and quality entry points.

Run schema-related tests:

cd ~/projects/zi-quant-platform
uv run pytest tests/test_services.py -k "schema or data_source or data_provenance"

If you clone this chapter from GitHub:

git clone https://github.com/ax2/zi-quant-platform.git
cd zi-quant-platform
git checkout chapter-05
uv sync --extra dev
uv run pytest

After starting the service, check health:

uv run uvicorn app.main:app --host 127.0.0.1 --port 8092
curl http://127.0.0.1:8092/health
curl http://127.0.0.1:8092/ready

If an API token is configured, later sync endpoints need X-Zi-Api-Token. This will be expanded when real data sync is implemented.

Stage Review For The First Five Articles

At this point, the first group of five articles completes a small loop.

Article 1 answers why programmers are suited to quant trading and sets the boundary: research, backtesting, paper trading, no real orders.

Article 2 builds the Python project skeleton, making ZiQuant installable, runnable, configurable, and testable.

Article 3 maps core concepts into models: stocks, stock universes, K-lines, factors, strategies, backtests, and paper trading.

Article 4 puts A-share trading rules into code and adds the testable trading_rules module.

Article 5 enters the data layer and emphasizes data-source abstraction, provenance, quality checks, and fallback marking.

These five articles do not keep repeating “quant trading is not mysticism”. They move the project from cognitive boundary to engineering skeleton, domain model, trading rules, and data-layer entry. The next group will continue along the data layer: PostgreSQL table design, stock master tables, a 500-stock A-share universe, real market-data integration, and data cleaning.

Chapter Update And Repository

This chapter adds:

• The rule that the data layer comes before the strategy layer.
• Data-source abstraction, market-data provenance, financial-report latency, fallback marking, and quality endpoints.
• A first-stage review of the first five articles, confirming consistent style, boundary, and project progress.

Repository:

https://github.com/ax2/zi-quant-platform

Code for this chapter:

git clone https://github.com/ax2/zi-quant-platform.git
cd zi-quant-platform
git checkout chapter-05
uv sync --extra dev
uv run pytest

Summary

Data comes before strategy.

Without provenance, quality checks, coverage, and fallback marking, strategy output has no trustworthy foundation. For programmers, the real starting point of quant trading is not writing a buy-sell formula. It is making the data pipeline replaceable, checkable, and reviewable.

The next article enters PostgreSQL and starts storing stocks, market bars, financial reports, and stock universes in stable tables.

WeChat

Follow ZiCode on WeChat

If this post was useful, you can follow later updates on WeChat as well.

X / Twitter

Follow @ax2_zicode

Faster technical notes, short thoughts, and new-post alerts are posted on X.

X Open X Profile @ax2_zicode