Feature Store Explained with a Real Example
What you build: Feast with real customer data: a leaky training set that scores 94 percent, the point-in-time truth, online serving and TTL. Every step tested.

✍️ Translate this blog into Hindi, Tamil or Telugu
A feature store sounds like yet another database. Its real job is narrower and more important: give a model the same feature values in training and in production, and give training only the values that were known at the time of each example. Get that wrong and your model looks brilliant in testing and fails in production. In this lab you use Feast (the open-source feature store) with a delivery app's customer data, build a training set the wrong way and the right way, see the difference in real numbers, then serve the latest values in milliseconds.
Who it is for: data scientists and ML engineers who keep hearing "feature store" and want to see what problem it solves. Time: about 45 minutes. Cost: one small VM for about 10 minutes, covered by the free trial credit.
Every command below was run for real by the CareerByteCode Team, and every output is the real result we got. The VM was created with gcloud from a Cloud Shell-like Linux machine; Feast ran on the VM over SSH.
Part A: feature stores in plain English

- Feature: an input to a model, for example "orders in the last 7 days" for a customer.
- Entity: who the feature is about (a customer, identified by
customer_id). - Feature view: a group of features from one source, with a TTL (how old a value may be and still count).
- Point-in-time join: for each training example (customer, date), take the feature values as they were on that date, never later ones.
- Offline store (here: Parquet files) for training; online store (here: SQLite) for fast lookups when the model serves.
- Materialize: copy the newest values from the offline store into the online store.
- Real-life example: judging a stock-picker by the newspaper of the day they picked, not by tomorrow's paper. Anyone looks brilliant with tomorrow's paper.
Before you start
A Google Cloud project with billing linked (the free trial works). We give Feast its own VM with 8 GB of memory, because its Python libraries are large; a 2 GB VM ran out of memory in our first try.
Part B: the lab
Step 1: a VM and Feast
# A VM for the lab with enough memory for the Python ML libraries (e2-standard-2: 2 vCPUs, 8 GB)
gcloud compute instances create feast-vm --zone=us-east1-b --machine-type=e2-standard-2 --image-family=debian-12 --image-project=debian-cloud --boot-disk-size=20GB --service-account=lab-shell@$PROJECT_ID.iam.gserviceaccount.com --scopes=cloud-platform > /dev/null 2>&1 && echo "feast-vm created"
until gcloud compute ssh feast-vm --zone=us-east1-b --tunnel-through-iap --command="true" > /dev/null 2>&1; do sleep 10; done; echo "SSH works"feast-vm created
SSH worksThe data: 30 days of stats for 200 customers of a delivery app, and labels saying whether the customer cancelled on 25 October. Every 10th customer cancelled, and after cancelling (from 26 October) their late deliveries jump, because the account is winding down. Save it as make_data.py:
# make_data.py: 30 days of daily stats for 200 customers of a delivery app (orders in the last 7 days,
# average basket, late deliveries), plus "label" rows: did the customer cancel their membership on that day?
import numpy as np
import pandas as pd
rng = np.random.default_rng(7)
days = pd.date_range("2026-10-01", periods=30, freq="D", tz="UTC")
rows = []
for cid in range(1, 201):
base = rng.uniform(1, 6)
for d in days:
late = int(rng.poisson(0.3 + (2 if cid % 10 == 0 and d.day > 25 else 0))) # every 10th customer: many late deliveries AFTER 25 Oct (after they cancelled)
rows.append({"customer_id": cid, "event_timestamp": d, "orders_7d": int(rng.poisson(base)),
"avg_basket": round(float(rng.normal(450, 80)), 2), "late_deliveries_7d": late})
stats = pd.DataFrame(rows)
stats.to_parquet("data/customer_stats.parquet", index=False)
labels = pd.DataFrame({"customer_id": range(1, 201)})
labels["event_timestamp"] = pd.Timestamp("2026-10-25", tz="UTC")
labels["cancelled"] = (labels.customer_id % 10 == 0).astype(int)
labels.to_parquet("data/labels.parquet", index=False)
print(f"customer_stats: {len(stats)} rows ({stats.customer_id.nunique()} customers x {len(days)} days), labels: {len(labels)} rows")# Feast (an open-source feature store) in a Python environment, and the example data: 200 customers x 30 days
sudo apt-get update -qq > /dev/null && sudo apt-get install -y -qq python3-venv > /dev/null 2>&1; python3 -m venv .venv && . .venv/bin/activate && pip install -q "feast>=0.40,<1" "pandas==2.*" "pyarrow" 2>&1 | grep -v notice | tail -1; feast version 2>/dev/null | tail -1
mkdir -p data && python3 make_data.pyFeast SDK Version: "0.68.0"
customer_stats: 6000 rows (200 customers x 30 days), labels: 200 rowsStep 2: define and register the features
The Feast settings (feature_store.yaml): the project name, where the registry lives, and SQLite as the online store:
# Feast settings: where feature definitions are kept (registry), and the online store for fast lookups
project: delivery
provider: local
registry: data/registry.db
online_store:
type: sqlite
path: data/online.db
entity_key_serialization_version: 3The definitions (features.py): one entity and one feature view with three features and a TTL of 2 days:
# features.py: the feature definitions. One entity (customer) and one feature view with three features,
# read from the daily stats file. ttl = how old a value may be and still count (here 2 days).
from datetime import timedelta
from feast import Entity, FeatureView, Field, FileSource
from feast.types import Float64, Int64
customer = Entity(name="customer", join_keys=["customer_id"])
stats_source = FileSource(path="data/customer_stats.parquet", timestamp_field="event_timestamp")
customer_stats = FeatureView(
name="customer_stats",
entities=[customer],
ttl=timedelta(days=2),
schema=[Field(name="orders_7d", dtype=Int64), Field(name="avg_basket", dtype=Float64), Field(name="late_deliveries_7d", dtype=Int64)],
source=stats_source,
online=True,
)feast apply imports every Python file in the folder to find definitions, so our two helper scripts must be listed in .feastignore (in our first run they were imported and crashed the apply):
make_data.py
train_check.py# Register the feature definitions: Feast reads features.py and writes them to its registry
. .venv/bin/activate; feast apply 2>&1 | grep -vi "warn" | tail -4
feast feature-views list 2>/dev/nullCreated feature view customer_stats
Created sqlite table delivery_customer_stats
NAME ENTITIES TYPE ENABLED STATE
customer_stats {'customer'} FeatureView Yes STATE_UNSPECIFIEDStep 3: a training set, the feature store way
Why: get_historical_features joins each label (customer, 25 October) with the stats as they were on 25 October:
# A training set the feature store way: for each label (customer, 25 Oct) take the stats AS THEY WERE on 25 Oct
. .venv/bin/activate
python3 -c "
import pandas as pd
from feast import FeatureStore
labels = pd.read_parquet('data/labels.parquet')
df = FeatureStore(repo_path='.').get_historical_features(entity_df=labels, features=['customer_stats:orders_7d', 'customer_stats:late_deliveries_7d']).to_df()
print(df[df.customer_id.isin([10, 11])].sort_values('customer_id').to_string(index=False))" 2>/dev/null customer_id event_timestamp cancelled orders_7d late_deliveries_7d
10 2026-10-25 00:00:00+00:00 1 2 0
11 2026-10-25 00:00:00+00:00 0 7 1
Customer 10 (who cancelled) had 0 late deliveries on 25 October. Nothing in their data before the cancellation hinted at it.
Break it: a training set with data from the future
The "quick" way: merge each customer's latest stats (from 30 October) onto the labels, then score a simple rule ("2 or more late deliveries = will cancel"). Save the scoring script as train_check.py:
# train_check.py: build a training set in two ways and score the same simple rule on each
# ("2 or more late deliveries in 7 days = will cancel").
# leaky : join each customer's LATEST stats (end of month), as a quick pandas merge would
# feast : point-in-time join, the stats as they were on the label date (25 Oct), via the feature store
import sys
import pandas as pd
from feast import FeatureStore
labels = pd.read_parquet("data/labels.parquet")
stats = pd.read_parquet("data/customer_stats.parquet")
mode = sys.argv[1]
if mode == "leaky":
latest = stats.sort_values("event_timestamp").groupby("customer_id").tail(1)
df = labels.merge(latest.drop(columns="event_timestamp"), on="customer_id")
used = latest.event_timestamp.max().date()
else:
store = FeatureStore(repo_path=".")
df = store.get_historical_features(entity_df=labels, features=["customer_stats:orders_7d", "customer_stats:late_deliveries_7d"]).to_df()
used = labels.event_timestamp.max().date()
acc = ((df.late_deliveries_7d >= 2).astype(int) == df.cancelled).mean()
print(f"{mode:<6} feature values from {used} | rule accuracy {100 * acc:.1f}% on {len(df)} customers")# Break it: the "quick" training set joins each customer's LATEST stats (30 Oct), after the label date. Score a simple rule
. .venv/bin/activate; python3 train_check.py leaky 2>/dev/nullleaky feature values from 2026-10-30 | rule accuracy 94.0% on 200 customers94 percent accuracy. It looks like a great signal, but it is leakage: late deliveries after 25 October are a consequence of cancelling, not a warning. In production, on the day you must predict, those values do not exist yet.
Fix it: the point-in-time training set
The same rule, on the values that were known on the label date:
# Fix it: the same rule on the point-in-time training set. The honest score is the one production will see
. .venv/bin/activate; python3 train_check.py feast 2>/dev/nullfeast feature values from 2026-10-25 | rule accuracy 86.0% on 200 customers86 percent, and that is actually worse than doing nothing: 90 percent of these customers did not cancel, so always guessing "no" scores 90 percent. The honest test shows the feature has no predictive value; the leaky one hid that completely. A model trained on the leaky set would have been shipped, and would have failed.

Step 4: online serving
Why: at prediction time the model needs the newest values for one customer, fast. We materialize October into the online store and look up two customers:
# Serving: copy the newest values into the online store (materialize), then read one customer in milliseconds
. .venv/bin/activate; feast materialize 2026-10-01T00:00:00 2026-11-01T00:00:00 2>&1 | grep -E "customer_stats|Materializing" | head -2
python3 -c "
import time
from feast import FeatureStore
s = FeatureStore(repo_path='.'); t0 = time.time()
r = s.get_online_features(features=['customer_stats:orders_7d', 'customer_stats:avg_basket', 'customer_stats:late_deliveries_7d'], entity_rows=[{'customer_id': 10}, {'customer_id': 11}]).to_dict()
print(r); print(f'online lookup took {1000 * (time.time() - t0):.1f} ms')" 2>/dev/nullMaterializing 1 feature views from 2026-10-01 00:00:00+00:00 to 2026-11-01 00:00:00+00:00 into the sqlite online store.
customer_stats:
{'customer_id': [10, 11], 'late_deliveries_7d': [4, 0], 'orders_7d': [3, 3], 'avg_basket': [317.52, 331.13]}
online lookup took 14.0 msThe same feature definitions serve training (offline) and production (online), so the model sees the same features in both: no training-serving skew.
Step 5: TTL: no stale values
Why: if a customer's data stopped arriving, an old value can be worse than none. We ask for a date (10 November) with no data in the 2 days before it:
# TTL: ask for features on a date with no data in the 2 days before it. Feast returns empty instead of a stale value
. .venv/bin/activate
python3 -c "
import pandas as pd
from feast import FeatureStore
e = pd.DataFrame({'customer_id': [10], 'event_timestamp': [pd.Timestamp('2026-11-10', tz='UTC')]})
print(FeatureStore(repo_path='.').get_historical_features(entity_df=e, features=['customer_stats:orders_7d']).to_df().to_string(index=False))" 2>/dev/null customer_id event_timestamp orders_7d
10 2026-11-10 00:00:00+00:00 <NA>
<NA>: Feast refuses to use a value older than the TTL. Your model or your code must handle missing features explicitly.
POC: prove it works
- Feast registered one entity and one feature view from Python definitions (Step 2).
- The point-in-time join gave the values known on 25 October (Step 3).
- The leaky set scored 94 percent; the honest set 86 percent, below always guessing "no" (Break it, Fix it).
- Online lookups returned the newest values in 14 ms, and the TTL refused a stale value (Steps 4 and 5).
Clean up (delete everything)
# Clean up: Feast keeps everything in local files on the VM, so deleting the VM removes it all
gcloud compute instances delete feast-vm --zone=us-east1-b --quiet 2>&1 | tail -1
echo "lab VMs left: $(gcloud compute instances list --filter='name=feast-vm' --format='value(name)' | wc -l)"Deleted [https://www.googleapis.com/compute/v1/projects/your-project-id/zones/us-east1-b/instances/feast-vm].
lab VMs left: 0FAQ
Do I need a feature store for every project? No. For one model and one team, careful SQL with point-in-time joins is enough. A feature store pays off when many models share features and when online serving must match training.
What does Google Cloud offer? Vertex AI Feature Store, which uses BigQuery as the offline store and serves online. Feast can also use BigQuery as its offline store. We used local files to keep the lab free; the concepts are the same.
What is the TTL for? To stop stale values from being used as if they were fresh. Set it from how often the source really updates.
Is leakage always this obvious? Rarely. It often hides in features computed "as of now" (lifetime totals, last status) or in joins on dates. A point-in-time join prevents most of it.
**Why did the first feast apply fail?** Feast imports every .py file in the repo folder. Keep helper scripts out of it, or list them in .feastignore.
Troubleshooting
| You see | Why | Fix |
|---|---|---|
Feature view ... does not exist after feast apply | The apply crashed while importing another Python file | Add helper scripts to .feastignore, run feast apply again |
| A training score that seems too good | Feature values from after the label date | Use a point-in-time join (get_historical_features) |
| Online lookups return None | Not materialized, or older than the TTL | Run feast materialize for the right dates; check the TTL |
No matching distribution found for feast==0.4* | That pattern matches only 0.4.x | Use a range such as feast>=0.40,<1 |
| Out of memory while installing or applying | Feast's dependencies are large | Use a VM with at least 4 to 8 GB |
What next
- The whole ML loop around a model: MLOps for Beginners: From Notebook to Production.
- Query the same kind of data in English: Text-to-SQL: Ask BigQuery Questions in Plain English.
- More hands-on posts: ByteBlog. Practice in ByteLabs and see the courses.
Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.
⭐ Keep learning with CareerByteCode
Hand-picked to go deeper on this topicGet CareerByteCode Team's next blog by email
One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.
We email you once to confirm. By subscribing you agree to the privacy policy.
Related blogs
CareerByteCode TeamMLflow on Google Cloud: Track Every Experiment
What you build: an MLflow tracking server on Google Cloud with models in Cloud Storage, five tracked runs, a registered champion and the missing-URI mistake. Every step tested.
#google-cloud
CareerByteCode TeamMLOps Project for Your Resume on Google Cloud
What you build: an MLOps loop on Google Cloud: quality gate, model registry, batch scoring to BigQuery, drift checks and a retrain that must earn its place. Every step tested.
#google-cloud
CareerByteCode TeamMLOps Interview Questions Answered with Labs
What you build: 6 MLOps interview answers proved with real scikit-learn experiments: leakage, reproducibility, imbalance, skew, versioning, latency. Every step tested.
#ci
CareerByteCode TeamCI/CD for Machine Learning with Cloud Build
What you build: a Cloud Build pipeline that retrains a churn model whenever new data lands in a bucket, checks it with a quality gate and redeploys on Cloud Run; broken data is stopped automatically.
💬 Comments (0)
Top · NewestSign in to comment