PC Lab 6 — Predicting Credit Scores with ML

The setting

You’re the new intern at JP Morgan, and the bank is going full digital: branches closing, and the plan on the table is to replace loan officers with AI. (The occupational-outlook data in the slides is blunt: loan officers’ automation risk — 92%.)

“We screen thousands of firms. I need an algorithm that reads a firm’s balance sheet and income statement and tells me its credit score — its default-risk class. Train it, prove it works, and send us predictions for the new applicants. We’ll score you on your true-positive ratio.”

Your goals: manipulate real firm-level accounting data, find out which variables actually predict credit risk, and build a classifier that could sit where a loan officer sits — then ask whether it should.

Background

This week’s lecture, Microeconomics of Banking, explains what you’re automating: screening under asymmetric information is the loan officer’s economic function. Toolkit: sklearn (and friends: xgboost, catboost, …).

The data

One zip, two CSVs of private-firm balance-sheet and income-statement data: Data_PCLab5_train_test.csv (labeled — train and test here) and Data_PCLab5_pred.csv (unlabeled — the “new applicants” you must score). (Yes, the files say PCLab5 — historical naming, this is Lab 6.)

Your assignment

Part 1 — Know your sample (Task 1)

• Import the train/test data. What are the different credit scores and what do they mean? Any useless variables?
• Plot variable distributions against credit scores; deal with outliers.
• Engineer features: build the firm’s age (year, year_creation) and any ratios you think a banker would look at.

Part 2 — What predicts risk? (Task 2 + optional)

• The Head of Risk wants the best predictors of credit scores: run a feature selection (SequentialFeatureSelector, SelectFromModel, or better) and interpret it.
• Optional: get feature importances from trees/random forests. Re-run the same model twice — results wobble. Why, and how do you stabilize them? How is feature importance different from feature selection?

Part 3 — The classifier (Task 3)

• Train and test an ML model of your choice; predict credit scores for Data_PCLab5_pred.csv.
• Submit your predictions as a .txt file — the TA scores them on the true-positive ratio.
• Bank policy accepts firms scoring above 4: what share of loan applications are you accepting?

Part 4 — Coarser classes (Task 4)

• The Head of Risk only wants a general risk profile: merge the scores into broader categories, retrain — does your TP ratio improve? What did you give up?

Part 5 — Beyond accuracy (Task 5)

• Accuracy is not welfare. Misclassifying a 3++ firm as 3 is harmless; denying credit to good firms — or funding bad ones — is not. That’s algorithmic discrimination territory.
• How often do your models badly misclassify? How would you modify the performance criterion to price in the social cost of errors?

Deliverables

• One Jupyter notebook per group (+ the predictions .txt), by email to the instructor (clement.mazetsonilhac@unibocconi.it) and the TA (andrea.andolfatto@phd.unibocconi.it).
• Email title — and notebook name: PCLab#6 - Group X - Name1 Name2 Name3. Tell us how long it took.
• Deadline: Friday, midnight (end of Week 7).

Grading

1. Submitting before the deadline;
2. Code quality — comments, readability, use of functions;
3. Notebook structure — well organized, explain what and why;
4. Completing the tasks and innovating — produce less, but more useful output;
5. Your predictions’ true-positive ratio on the held-out firms.