Imports#

Only the standard Python data science packages and the specialized macrosynergy package are needed.

import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import math

import json
import yaml

import macrosynergy.management as msm
import macrosynergy.panel as msp
import macrosynergy.signal as mss
import macrosynergy.pnl as msn


from macrosynergy.download import JPMaQSDownload

from timeit import default_timer as timer
from datetime import timedelta, date, datetime

import warnings

warnings.simplefilter("ignore")

The JPMaQS indicators we consider are downloaded using the J.P. Morgan DataQuery API interface within the macrosynergy package. This is done by specifying ticker strings, formed by appending an indicator category code <category> to a currency area code <cross_section>. These constitute the main part of a full quantamental indicator ticker, taking the form DB(JPMAQS,<cross_section>_<category>,<info>), where <info> denotes the time series of information for the given cross-section and category.

The following types of information are available:

  • value giving the latest available values for the indicator

  • eop_lag referring to days elapsed since the end of the observation period

  • mop_lag referring to the number of days elapsed since the mean observation period

  • grade denoting a grade of the observation, giving a metric of real time information quality.

After instantiating the JPMaQSDownload class within the macrosynergy.download module, one can use the download(tickers, start_date, metrics) method to obtain the data. Here tickers is an array of ticker strings, start_date is the first release date to be considered and metrics denotes the types of information requested.

# Cross-sections of interest
cids_dmca = [
    "AUD",
    "CAD",
    "CHF",
    "EUR",
    "GBP",
    "JPY",
    "NOK",
    "NZD",
    "SEK",
    "USD",
]  # DM currency areas
cids_dmec = ["DEM", "ESP", "FRF", "ITL", "NLG"]  # DM euro area countries
cids_latm = ["BRL", "COP", "CLP", "MXN", "PEN"]  # LatAm countries
cids_emea = ["CZK", "HUF", "ILS", "PLN", "RON", "RUB", "TRY", "ZAR"]  # EMEA countries
cids_emas = [
    "CNY",
    "IDR",
    "INR",
    "KRW",
    "MYR",
    "PHP",
    "SGD",
    "THB",
    "TWD",
]  # EM Asia countries (without "HKD")

cids_dm = cids_dmca + cids_dmec
cids_em = cids_latm + cids_emea + cids_emas

cids = sorted(cids_dm + cids_em)
# Quantamental categories of interest
main = [
    # Merchandise Trade Balances:
    "MTBGDPRATIO_NSA_12MMA_D1M1ML3",
    "MTBGDPRATIO_SA_3MMA_D1M1ML3",
    "MTBGDPRATIO_SA_6MMA_D1M1ML6",
    "MTBGDPRATIO_SA_3MMAv24MMA",
    "MTBGDPRATIO_SA_3MMAv60MMA",
    "MTBGDPRATIO_SA_3MMAv120MMA",
    "MTB_SA_3MMAvEWMAZ",
    # Basical External balances:
    "BXBGDPRATIO_NSA_12MMA_D1M1ML3",
    "BXBGDPRATIO_NSA_12MMAv60MMA",
    "BXBGDPRATIO_NSA_12MMAv120MMA",
    # Seasonally adjusted current account balances:
    "CABGDPRATIO_SA_3MMA_D1M1ML3",
    "CABGDPRATIO_SA_6MMA_D1M1ML6",
    "CABGDPRATIO_SA_3MMAv60MMA",
    "CABGDPRATIO_SA_1QMA_D1Q1QL1",
    "CABGDPRATIO_SA_2QMA_D1Q1QL2",
    "CABGDPRATIO_SA_1QMAv20QMA",
]

# economic context
econ = [
    "INTRGDP_NSA_P1M1ML12_3MMA",
    "BXBGDPRATIO_NSA_12MMA",
    "EXMOPENNESS_NSA_1YMA",
]

# market links
mark = [
    "FXXR_NSA",
    "DU05YXR_VT10",
    "DU05YCRY_VT10",
    "FXTARGETED_NSA",
    "FXUNTRADABLE_NSA",
]

xcats = main + econ + mark
# Download series from J.P. Morgan DataQuery by tickers
start_date = "1995-01-01"
tickers = [cid + "_" + xcat for cid in cids for xcat in xcats]
print(f"Maximum number of tickers is {len(tickers)}")

# Retrieve credentials

client_id: str = os.getenv("DQ_CLIENT_ID")
client_secret: str = os.getenv("DQ_CLIENT_SECRET")

# Download from DataQuery

with JPMaQSDownload(client_id=client_id, client_secret=client_secret) as downloader:
    dfd = downloader.download(
        tickers=tickers,
        start_date=start_date,
        metrics=["value", "eop_lag", "mop_lag", "grading"],
        suppress_warning=True,
        show_progress=True,
        report_time_taken=True,
    )
Maximum number of tickers is 888
Downloading data from JPMaQS.
Timestamp UTC:  2023-07-14 09:05:52
Connection successful!
Number of expressions requested: 3552
Requesting data: 100%|███████████████████████████████████████████████████████████████| 178/178 [00:55<00:00,  3.20it/s]
Downloading data: 100%|██████████████████████████████████████████████████████████████| 178/178 [02:28<00:00,  1.20it/s]
Time taken to download data: 	205.02 seconds.
Time taken to convert to dataframe: 	51.67 seconds.

Availability#

cids_exp = sorted(list(set(cids) - set(cids_dmec)))  # cids expected in category panels
msm.missing_in_df(dfd, xcats=main, cids=cids_exp)
Missing xcats across df:  set()
Missing cids for BXBGDPRATIO_NSA_12MMA_D1M1ML3:  set()
Missing cids for BXBGDPRATIO_NSA_12MMAv120MMA:  set()
Missing cids for BXBGDPRATIO_NSA_12MMAv60MMA:  set()
Missing cids for CABGDPRATIO_SA_1QMA_D1Q1QL1:  {'EUR', 'PLN', 'CZK', 'KRW', 'THB', 'GBP', 'RON', 'TRY', 'PHP', 'BRL', 'JPY'}
Missing cids for CABGDPRATIO_SA_1QMAv20QMA:  {'EUR', 'PLN', 'CZK', 'KRW', 'THB', 'GBP', 'RON', 'TRY', 'PHP', 'BRL', 'JPY'}
Missing cids for CABGDPRATIO_SA_2QMA_D1Q1QL2:  {'EUR', 'PLN', 'CZK', 'KRW', 'THB', 'GBP', 'RON', 'TRY', 'PHP', 'BRL', 'JPY'}
Missing cids for CABGDPRATIO_SA_3MMA_D1M1ML3:  {'TWD', 'SEK', 'RON', 'PEN', 'RUB', 'CHF', 'MXN', 'AUD', 'CAD', 'NOK', 'SGD', 'HUF', 'ILS', 'CNY', 'COP', 'CLP', 'INR', 'USD', 'MYR', 'NZD', 'ZAR', 'IDR'}
Missing cids for CABGDPRATIO_SA_3MMAv60MMA:  {'TWD', 'SEK', 'RON', 'PEN', 'RUB', 'CHF', 'MXN', 'AUD', 'CAD', 'NOK', 'SGD', 'HUF', 'ILS', 'CNY', 'COP', 'CLP', 'INR', 'USD', 'MYR', 'NZD', 'ZAR', 'IDR'}
Missing cids for CABGDPRATIO_SA_6MMA_D1M1ML6:  {'TWD', 'SEK', 'RON', 'PEN', 'RUB', 'CHF', 'MXN', 'AUD', 'CAD', 'NOK', 'SGD', 'HUF', 'ILS', 'CNY', 'COP', 'CLP', 'INR', 'USD', 'MYR', 'NZD', 'ZAR', 'IDR'}
Missing cids for MTBGDPRATIO_NSA_12MMA_D1M1ML3:  set()
Missing cids for MTBGDPRATIO_SA_3MMA_D1M1ML3:  set()
Missing cids for MTBGDPRATIO_SA_3MMAv120MMA:  set()
Missing cids for MTBGDPRATIO_SA_3MMAv24MMA:  set()
Missing cids for MTBGDPRATIO_SA_3MMAv60MMA:  set()
Missing cids for MTBGDPRATIO_SA_6MMA_D1M1ML6:  set()
Missing cids for MTB_SA_3MMAvEWMAZ:  set()

Real-time quantamental indicators of external ratios are available back to the 1990s for most developed countries and the early 2000s for some emerging economies.

For the explanation of currency symbols, which are related to currency areas or countries for which categories are available, please view Appendix 3.

xcatx = main
cidx = cids_exp

dfx = msm.reduce_df(dfd, xcats=xcatx, cids=cidx)
dfs = msm.check_startyears(dfx)
msm.visual_paneldates(dfs, size=(18, 8))

print("Last updated:", date.today())
../_images/External ratios trends_24_0.png
Last updated: 2023-07-14
plot = msm.check_availability(
    dfd, xcats=xcatx, cids=cidx, start_size=(18, 8), start_years=False, start=start_date
)
../_images/External ratios trends_25_0.png

Vintage grades are mixed, with developed markets on average recording better grades than EM countries. The higher grades of seasonally-adjusted series reflect the sequential seasonal adjustment that allows deriving a good proxy of real-time data even from non-original vintages.

plot = msp.heatmap_grades(
    dfd,
    xcats=xcatx,
    cids=cidx,
    start=start_date,
    size=(18, 8),
    title=f"Average vintage grades, from {start_date} onwards",
)
../_images/External ratios trends_27_0.png
mtbs = [
    "MTBGDPRATIO_NSA_12MMA_D1M1ML3",
    "MTBGDPRATIO_SA_3MMA_D1M1ML3",
    "MTB_SA_3MMAvEWMAZ",
]
msp.view_ranges(
    dfd,
    xcats=mtbs,
    cids=cids_exp,
    val="eop_lag",
    title="End of observation period lags (ranges of time elapsed since end of observation period in days)",
    start="2000-01-01",
    kind="box",
    size=(16, 4),
)
msp.view_ranges(
    dfd,
    xcats=mtbs,
    cids=cids_exp,
    val="mop_lag",
    title="Median of observation period lags (ranges of time elapsed since middle of observation period in days)",
    start="2000-01-01",
    kind="box",
    size=(16, 4),
)
../_images/External ratios trends_28_0.png ../_images/External ratios trends_28_1.png
cabs = [
    "CABGDPRATIO_SA_3MMA_D1M1ML3",
    "CABGDPRATIO_SA_6MMA_D1M1ML6",
    "CABGDPRATIO_SA_3MMAv60MMA",
    "CABGDPRATIO_SA_1QMA_D1Q1QL1",
    "CABGDPRATIO_SA_2QMA_D1Q1QL2",
    "CABGDPRATIO_SA_1QMAv20QMA",
]
msp.view_ranges(
    dfd,
    xcats=cabs,
    cids=cids_exp,
    val="eop_lag",
    title="End of observation period lags (ranges of time elapsed since end of observation period in days)",
    start="2000-01-01",
    kind="box",
    size=(16, 4),
)
msp.view_ranges(
    dfd,
    xcats=cabs,
    cids=cids_exp,
    val="mop_lag",
    title="Median of observation period lags (ranges of time elapsed since middle of observation period in days)",
    start="2000-01-01",
    kind="box",
    size=(16, 4),
)
../_images/External ratios trends_29_0.png ../_images/External ratios trends_29_1.png
xbbs = [
    "BXBGDPRATIO_NSA_12MMA_D1M1ML3",
    "BXBGDPRATIO_NSA_12MMAv60MMA",
    "BXBGDPRATIO_NSA_12MMAv120MMA",
]
msp.view_ranges(
    dfd,
    xcats=xbbs,
    cids=cids_exp,
    val="eop_lag",
    title="End of observation period lags (ranges of time elapsed since end of observation period in days)",
    start="2000-01-01",
    kind="box",
    size=(16, 4),
)
msp.view_ranges(
    dfd,
    xcats=xbbs,
    cids=cids_exp,
    val="mop_lag",
    title="Median of observation period lags (ranges of time elapsed since middle of observation period in days)",
    start="2000-01-01",
    kind="box",
    size=(16, 4),
)
../_images/External ratios trends_30_0.png ../_images/External ratios trends_30_1.png
# Renaming extended categories for ease of subsequent analysis
rename_dict = {
    "CABGDPRATIO_SA_1QMA_D1Q1QL1": "CABGDPRATIO_SA_3MMA_D1M1ML3",
    "CABGDPRATIO_SA_2QMA_D1Q1QL2": "CABGDPRATIO_SA_6MMA_D1M1ML6",
    "CABGDPRATIO_SA_1QMAv20QMA": "CABGDPRATIO_SA_3MMAv60MMA",
}
dfd["xcat"] = dfd["xcat"].map(lambda x: rename_dict.get(x, x))

History#

Importance#

Empirical clues#

dfb = dfd[dfd["xcat"].isin(["FXTARGETED_NSA", "FXUNTRADABLE_NSA"])].loc[
    :, ["cid", "xcat", "real_date", "value"]
]
dfba = (
    dfb.groupby(["cid", "real_date"])
    .aggregate(value=pd.NamedAgg(column="value", aggfunc="max"))
    .reset_index()
)
dfba["xcat"] = "FXBLACK"
fxblack = msp.make_blacklist(
    dfba, "FXBLACK"
)  # exclude periods of FX trageting or illiquidity

There has been a clear negative correlation between increases in merchandise trade balance ratios (shifts towards surplus) and economic growth trends. High-growth countries more often display a shift towards a deficit. Low-growth countries more often display a shift towards a surplus. In conjunction, these two indicators can provide a rough characterization of the macro state of economies. High-growth and rising-deficit countries often experience overheating and subsequent policy tightening, whereas low-growth and rising-surplus countries often lack policy stimulus and subsequently enjoy accommodative monetary policy.

xcatx = ["MTBGDPRATIO_NSA_12MMA_D1M1ML3", "INTRGDP_NSA_P1M1ML12_3MMA"]
cids_fx = sorted(list(set(cids_exp) - set(["EUR", "JPY", "USD"])))

cr = msp.CategoryRelations(
    dfd,
    xcats=["MTBGDPRATIO_NSA_12MMA_D1M1ML3", "INTRGDP_NSA_P1M1ML12_3MMA"],
    cids=cids_fx,
    freq="M",
    lag=0,
    xcat_aggs=["mean", "mean"],
    start=start_date,
    years=5,
)
cr.reg_scatter(
    title="Trade balance ratio trend and GDP growth trend across countries and half-decades",
    labels=True,
    coef_box="upper right",
    ylab="GDP growth trend, % over a year ago (intuitive growth metric)",
    xlab="Merchandise trade balance ratio as % of GDP, change over preceding 3 months in 12-month moving average",
)
../_images/External ratios trends_73_0.png

The negative correlation between merchandise trade balance ratios and GDP growth trends can be exploited

cids_eur = ["CHF", "CZK", "HUF", "NOK", "PLN", "RON", "SEK"]  # EUR benchmark cids
cids_eud = ["GBP", "TRY", "RUB"]  # mean(EUR,USD) benchmark cids
cids_usd = list(set(cidx) - set(cids_eur) - set(cids_eud))  # USD benchmark cids

xcatx = [
    "INTRGDP_NSA_P1M1ML12_3MMA",
]


for xc in xcatx:
    calc_eur = [f"{xc}vBM = {xc} - iEUR_{xc}"]
    calc_usd = [f"{xc}vBM = {xc} - iUSD_{xc}"]
    calc_eud = [f"{xc}vBM = {xc} - 0.5 * ( iEUR_{xc} + iUSD_{xc} )"]

    dfa_eur = msp.panel_calculator(dfd, calcs=calc_eur, cids=cids_eur)
    dfa_usd = msp.panel_calculator(dfd, calcs=calc_usd, cids=cids_usd)
    dfa_eud = msp.panel_calculator(dfd, calcs=calc_eud, cids=cids_eud)

    dfa = pd.concat([dfa_eur, dfa_usd, dfa_eud])
    dfd = msm.update_df(dfd, dfa)
# Compute Zn scores relative to zero for both gdp growth and trade balance changes

xcatx = [
    # Merchandise trade balances
    "MTBGDPRATIO_NSA_12MMA_D1M1ML3",
    "MTBGDPRATIO_SA_6MMA_D1M1ML6",
    "MTBGDPRATIO_SA_3MMAv24MMA",
    "MTBGDPRATIO_SA_3MMAv60MMA",
    # Growth
    "INTRGDP_NSA_P1M1ML12_3MMAvBM",
]

for xcat in xcatx:
    dfa = msp.make_zn_scores(
        df=dfd,
        cids=list(cidx),
        xcat=xcat,
        start=start_date,
        est_freq="m",
        neutral="zero",
        thresh=4,
    )
    dfd = msm.update_df(dfd, dfa)
calcs = [
    "GDP_MTDv24_ZSC = 1/2 * ( MTBGDPRATIO_SA_3MMAv24MMAZN +  INTRGDP_NSA_P1M1ML12_3MMAvBM ) "
]

dfa = msp.panel_calculator(dfd, calcs=calcs, cids=cidx)
dfd = msm.update_df(dfd, dfa)
xcatx = ["GDP_MTDv24_ZSC", "FXXR_NSA"]
cidx = list(set(cids_exp).intersection(cids_dm) - set(["USD"]))

cr = msp.CategoryRelations(
    dfd,
    xcats=xcatx,
    cids=cidx,
    freq="Q",
    lag=1,
    xcat_aggs=["last", "sum"],
    # xcat_trims=[20, 40],
    blacklist=fxblack,
    start=start_date,
    years=None,
)

cr.reg_scatter(
    title="Composite merchandise trade balance and GDP growth trends, and subsequent quarterly FX forward returns, since 2000",
    labels=False,
    coef_box="upper right",
    xlab="Average of MTB growth and GDP growth vs benchmark zn-scores, last value of quarter",
    ylab="Cumulative FX forward returns, next-quarter",
)
../_images/External ratios trends_78_0.png

Seasonally-adjusted current account changes#

Current account shifts towards lower deficits or higher surpluses increase, all other things equal, prospects for capital inflows and currency appreciation. Both bode well for the idiosyncratic performance of local fixed income markets. The below analysis shows some evidence that quarterly changes in current account ratios, multiplied with openness (ratio of exports and imports to GDP) positively predict subsequent IRS fixed-receiver returns relative to the U.S. market.

cidx = list(set(cids) - set(["USD"]))
calcs = [
    "CAB_IMPACT = EXMOPENNESS_NSA_1YMA * CABGDPRATIO_SA_3MMA_D1M1ML3",
    "DU05YXR_VT10vUSD = DU05YXR_VT10 - iUSD_DU05YXR_VT10",
]

dfa = msp.panel_calculator(dfd, calcs=calcs, cids=cidx)
dfd = msm.update_df(dfd, dfa)
xcatx = ["CAB_IMPACT", "DU05YXR_VT10vUSD"]
cidx = list(
    set(cids)
    - set(["BRL", "DEM", "ESP", "FRF", "ITL", "NLG", "PEN", "PHP", "RON", "USD"])
)  # missing cids from CAB_IMPACT or DU05YXR_VT10vUSD

cr = msp.CategoryRelations(
    dfd,
    xcats=xcatx,
    cids=cidx,
    freq="Q",
    lag=1,
    xcat_aggs=["last", "sum"],
    xcat_trims=[25, 100],  # exclude invalid outliers
    blacklist=fxblack,
    start=start_date,
    years=None,
)

cr.reg_scatter(
    title="Current account changes and subsequent duration returns (versus USD), EM and DM since 2000",
    labels=False,
    coef_box="lower right",
    xlab="Change in current account ratio (sa) to GDP, 3mma over previous 3mma, adjusted for openness quarter end",
    ylab="5-year IRS receiever returns (10%ar vol targeted) versus USD, next quarter",
    prob_est="map",
)
../_images/External ratios trends_82_0.png

Appendices#

Appendix 1: Seasonal adjustment procedure#

We deploy the state-of-the-art US Census X-13 algorithm to handle seasonality in economic series. There are two characteristics of the model worth noting in connection to the point-in-time estimation JPMaQS is delivering:

  1. The underlying econometric model is a seasonal ARIMA model leveraging a two-sided filter. As both the earliest and latest observations are included in the historical seasonal component estimation, JPMaQS forces the model (and seasonal component) to being re-estimated for each release date’s time-series. This ensures that only information available at a given point-in-time is used, avoiding leakage of future information.

  2. The algorithm has an embedded endogenous model specification using an Akaike Information Criteria, ensuring the parameters are dynamically chosen using only information available at each point-in-time.

For our seasonally adjustment of the Merchandise Trade Balances, we sequentially apply the multiplicative method of the U.S. Census X-13 algorithm to the levels of imports and exports respectively for each of their vintages. We then subtract the imports from the exports to get the seasonally adjusted merchandise trade balances, and take ratios to nominal GDP at the end. This contrast with the Current Account Balances, were we first take the ratio to GDP to deal deal with non-stationarity in the data, and then apply the additive X-13 method to these ratios for each release date. As current account balances can be both in surplus and deficit, we cannot use the multiplicative method to deal with the non-stationarity of the data.

Appendix 2: Notes on OECD data integration#

Some indicators in this notebook are constructed using vintages provided by the OECD’s Revision Analysis Dataset in addition to national sources series’. The integration of the OECD datasets follows the following rules:

  1. The following priority order is applied for combining vintages. First, JPMaQS uses seasonally and calendar adjusted original vintages from national sources. Beyond that JPMaQS uses OECD vintages.

  2. OECD vintages inform on the month of release but not the exact date. Actual release dates for these vintages are estimated based on release days of subsequent vintages.

  3. Inconsistencies, data errors and missing values in the OECD vintages have been corrected for JPMaQS.

OECD data is seasonally adjusted and denominated in native currency. No such consistency is found in national sources. Often figures are stated in a foreign denomination (EUR or USD) and no seasonal adjustment has been applied. To integrate these two data sources we first currency convert national sources into their native currency and then apply seasonal adjustment.

Appendix 3: Currency symbols#

The word ‘cross-section’ refers to currencies, currency areas or economic areas. In alphabetical order, these are AUD (Australian dollar), BRL (Brazilian real), CAD (Canadian dollar), CHF (Swiss franc), CLP (Chilean peso), CNY (Chinese yuan renminbi), COP (Colombian peso), CZK (Czech Republic koruna), DEM (German mark), ESP (Spanish peseta), EUR (Euro), FRF (French franc), GBP (British pound), HKD (Hong Kong dollar), HUF (Hungarian forint), IDR (Indonesian rupiah), ITL (Italian lira), JPY (Japanese yen), KRW (Korean won), MXN (Mexican peso), MYR (Malaysian ringgit), NLG (Dutch guilder), NOK (Norwegian krone), NZD (New Zealand dollar), PEN (Peruvian sol), PHP (Philippine peso), PLN (Polish zloty), RON (Romanian leu), RUB (Russian ruble), SEK (Swedish krona), SGD (Singaporean dollar), THB (Thai baht), TRY (Turkish lira), TWD (Taiwanese dollar), USD (U.S. dollar), ZAR (South African rand).