Skip to content

Tushare Data Downloads ​

The built-in download_tushare_task is an api Task that organizes request results into Parquet files in the workspace. It prepares raw data; the ETL plugin generates Alpha158 features.

Configuring the data connection ​

Set these variables in the service environment executing the Task:

bash
export AXONX_TUSHARE_TOKEN='<data API token>'
# Direct connection to your own compatible upstream (not an AxonX proxy)
export AXONX_TUSHARE_BASE_URL='http://<data service>/dataapi'

The built-in client defaults to http://api.waditu.com/dataapi. Requests use <base_url>/<api_name> and submit api_name, token, params, and fields as JSON. Custom services must support this protocol. This token and the AXONX_SERVICE_TOKEN protecting the AxonX HTTP service are separate credentials.

When connecting through an AxonX proxy named tushare, use the proxy root instead:

bash
export AXONX_TUSHARE_BASE_URL='http://<AxonX proxy host>:1024/proxy/tushare'

The client appends /trade_cal or another API name. /tushare alone is not the AxonX proxy route; the /proxy prefix is required. On the proxy service, components.proxy.tushare.upstream_base_url points to the real upstream. The commented default configuration obtains that upstream from AXONX_TUSHARE_BASE_URL, while download clients use the same variable name for their destination. Keep these process environments separate; if the proxy and Task share one service, use a separate variable for the configured upstream as shown in HTTP proxy.

Access scope, quotas, and authorization are determined upstream; this page only describes current client and task behavior. For an AxonX proxy, see HTTP proxy. Do not assume that replacing the URL guarantees protocol compatibility.

Parameters and dates ​

Open Development tools → Task definitions and select download_tushare_task. The form displays dates, calendar-day lookback, request timeout, and dataset groups according to the current task Schema. See Using Studio for the complete task-definition interface. Confirm data credentials on the execution machine before running the Task.

FieldDefaultPurpose
start_dateEmptyStart date, inclusive
end_dateEmptyEnd date; empty means today locally, future dates are capped at today
days_back7Calendar-day lookback when no start date is given
timeout600 secondsTimeout per network request, must be greater than zero
datasetsAll five groupsComma-separated string; also accepts a JSON list of strings

Dates accept YYYYMMDD and ISO formats; eight-digit integer dates from the CLI are converted back to strings. Date ranges enumerate calendar days. No file is written when a holiday request returns empty data. days_back counts calendar days, not trading days.

bash
axonx exec --task download_tushare_task --days-back 7
axonx exec --task download_tushare_task \
  --start-date 20230101 --end-date 20231231 \
  --datasets 'static,stk_limit,daily,adj_factor,index_weight'

Exec waits for completion. Inspect its exit code and full metadata; for separate observation use wait_task with the Task ID. Long downloads require independent API and client timeout settings.

Dataset groups ​

Optional groupQuery contentOutput location
staticstock_basic, namechange, trade_caltushare/ root
stk_limitOfficial price limitsCorresponding date partition
dailyUnadjusted daily market dataCorresponding date partition
adj_factorAdjustment factorsCorresponding date partition
index_weightCSI 300 constituent weights, 000300.SHWeight record date partition

static queries stock statuses L, D, P, and G, merges the results, and removes duplicates. Static files are snapshots at query time; the stock_basic snapshot itself is not complete historical constituent data. Historical name changes come separately from namechange.

Weight queries call the API for covered months; month boundaries may extend beyond the requested start and end dates. Results are partitioned by returned weight dates rather than copied to every trading day.

To update only market data:

bash
axonx exec --task download_tushare_task \
  --start-date 20240101 --end-date 20240131 \
  --datasets 'daily,adj_factor'

Groups can be selected independently, but qlib_a158 ETL requires at least market data, adjustments, a trading calendar, stock master data, and historical names. Official price limits and index weights affect tradability and benchmark interpretation.

Output directory ​

Open the tushare/ folder in Studio's Workspace page and select a static file or date partition. The right side displays its Parquet Schema and paginated data; previewing does not mean the full table has been loaded.

Example using the default .axonx/ workspace:

text
.axonx/
  tushare/
    stock_basic.parquet
    namechange.parquet
    trade_cal.parquet
    2023/
      20230103/
        daily.parquet
        adj_factor.parquet
        stk_limit.parquet
      20230131/
        index_weight.parquet

Nonempty responses are checked against declared fields, stably sorted, then atomically written as Parquet with zstd compression. Downloading the same file path again replaces the existing file. Empty responses skip writes and do not actively delete existing old files.

Download task metadata output_params includes start_date, end_date, files, and per-dataset rows counts. files is a list of string paths actually written, not the within-directory artifacts mapping used by standard research tasks.

Completeness and failures ​

The client performs limited retries for network errors and specified transient errors. Rate limits have a separate budget; logs show wait times and retry counts. Invalid JSON, invalid table structures, or errors not declared as supported cause failure.

Static queries use pagination, deduplicate page results, and detect repeated pages and maximum page counts. Daily and weight queries require complete single-request results. A response with has_more=true raises an error to avoid treating truncated data as a complete partition.

SymptomChecks and actions
Upstream error or authorization failureToken, compatible URL, and data service permissions
Long retry waitsRate-limit reasons and retry budget in task logs
Zero rows for a dataset in metadataWhether the group was selected, whether the date was a trading day, and whether upstream returned an empty set
ETL lacks master dataDownload static separately and check all three static files
Insufficient ETL historyDownload earlier history; the latest 7 days cannot support multi-year training

Raw data lives outside task directories in tushare/. When migrating a research environment, back up the data root separately, or first generate and save the data needed for research as ETL task artifacts.

Agent-native quant research.