PC Lab 4 — Predicting Stock Returns with ML

The setting

The big league. You’re the new intern at Renaissance Technologies LLC, and Jim Simons has read Gu, Kelly & Xiu (2020) and Jiang et al. (2023).

“They predict returns with machine learning at monthly frequency and hundreds of signals. I want to know what happens at daily frequency with only two signals — prices and volumes, like Jiang et al. Try their algorithms. Then don’t just show me R-squareds: run the money and tell me what a hundred dollars becomes.”

Your goals: build proper train/test pipelines for return prediction, compare linear models against ML honestly (out-of-sample!), and discover what every quant learns the hard way — the gap between a good backtest and a good net-of-fees backtest.

Background

This week’s lecture, Asset Pricing with Big Data, covers ML in asset pricing. The academic anchor is Gu, Kelly & Xiu (2020), Empirical Asset Pricing via Machine Learning — your out-of-sample R² formula comes from there (eq. 19, p. 2246).

The data

Two CSVs, same 8 stocks + S&P 500 as always, daily from 2012: prices and trading volumes (columns Date, AAPL, BA, T, MGM, AMZN, IBM, TSLA, GOOG, sp500). Or pull your own from Yahoo Finance.

Your assignment

Part 1 — Know your sample (Task 1)

Import prices and volumes. Average trading volume for Apple? Maximum for the S&P 500? Which security is traded the most — and why does that make sense?
Plot the time series of volumes (raw and normalized).
Is there a correlation between returns and changes in volume?

Part 2 — Linear predictions (Task 2)

Build one dataframe of dates, prices, and volumes (tip: scale with MinMaxScaler).
Predict returns over short (5-day), medium (20-day), and long (60-day) horizons from past returns and volumes over the past 5, 20, and 60 days.
Split 75% training / 25% testing. Train an OLS model and a Ridge regression (play with the penalty).
Evaluate with the out-of-sample R² of Gu et al. (2020) — plus any alternative metric you like (sign hit-rate, MSE).
Improve it (Task 2 bis): what happens if you add the market as a signal? Or lagged returns?

Part 3 — Bring in the machines (Task 3)

Pick one or more ML models — a multi-layer neural network, trees — and repeat Part 2.
Which method wins? Is that in line with Gu et al. (2020)? Can you really compare your setting to theirs? What about Jiang et al. (2023)?

Part 4 — Run the money (Task 4)

Build an AI-driven long portfolio: every rebalancing date, hold the 4 assets with the highest predicted return. Invest $100 at the start of the test period — how much do you end with?
Benchmark against 1,000 random-weight portfolios (weights fixed at the start, never rebalanced).
Now add trading fees of 3 bps per transaction. What’s the performance now — and what does that teach you about high-frequency rebalancing?

Deliverables

One Jupyter notebook per group, by email to the instructor (clement.mazetsonilhac@unibocconi.it) and the TA (andrea.andolfatto@phd.unibocconi.it).
Email title — and notebook name: PCLab#4 - Group X - Name1 Name2 Name3. Tell us how long it took.
Deadline: Friday, midnight (end of Week 5).

Grading

Submitting before the deadline;
Code quality — comments, readability, use of functions;
Notebook structure — well organized, explain what and why;
Completing the tasks and innovating — produce less, but more useful output.

These labs also build your project portfolio — they are great material for CVs and job interviews.