Agricultural Predictive Pipeline · Sri Lanka

Rice Production
Forecasting Project

A statistically rigorous time-series forecasting and machine learning pipeline, predicting crop outputs across Maha & Yala seasons using SARIMAX, Random Forest, and Residual Hybrid Models.

🌾 Agriculture Time Series Forecasting SARIMAX (1,1,1)(1,1,0,2) Hybrid Residual Coupling Spearman Rank Test Robust Scaling
Explore the Analysis → ML Pipeline Blueprint
149
Historical Seasons (1950–2024)
8
Predictive Features
6.67%
SARIMAX MAPE
0.913
Random Forest R² Score
Project Context

The Challenge of Food Security

Rice cultivation is the backbone of Sri Lanka\'s food supply. Accurate forecasting guides imports, pricing policy, and agricultural planning, safeguarding against monsoon-driven crop failures.

🌦️
2
Major Cultivation Seasons (Yala, Maha)
🚜
90.4%
Target-Harvested Acres Correlation
📈
74.5
Years of Historical Records
🍲
100%
Staple Food Security Focus
🧭 Robust Preprocessing Focus

Traditional models drop outliers or use random splits. This project strictly prevents data leakage by implementing chronological training/testing splits and handles outliers using Robust Scaling to keep sequential series indexing intact.

Dataset Schema

Feature Glossary

Economic, environmental, and climatic feature vectors mapped from census records.

Feature Name Type Description Monotonic correlation (r)
Year_New Temporal Chronological Year (1950 - 2024) Trended
Season Categorical Cultivating cycle (Yala = southwest monsoon, Maha = northeast monsoon) Strong seasonal driver
Sown (*000 Acres) Continuous Total land area sown with paddy seed +0.887 (Strong positive)
Harvested (*000 Acres) Continuous Total land area successfully harvested +0.904 (Strongest positive)
GDP B$ Continuous Sri Lanka GDP in billions of USD +0.835
Inflation(%) Continuous Annual inflation rate +0.337
Rainfall(mm) Continuous Mean seasonal cumulative precipitation +0.365
Temperature(°C) Continuous Mean seasonal temperature -0.295 (Negative correlation)
Production (*000 Mt.) 🌾 Continuous Total paddy crop harvested (target variable) Target
Model Performance

Model Evaluation & Benchmarks

Chronological split validation (Train: 1950Q2–2018Q4, Test: 2019Q2–2024Q2) preventing data leakage.

Analytical Section 10
Performance Metrics on Out-of-Sample Test Set
SARIMA vs SARIMAX (with exog forecasting) vs Machine Learning vs Residual Hybrid
Model Pipeline Validation strategy R² Score RMSE (Mt) MAPE (%)
Baseline SARIMA (1,1,1)(1,1,0,2) Chronological Split 0.5401 238.45 8.12%
Advanced SARIMAX (1,1,1)(1,1,0,2) Chronological + Exog Forecasted 0.8924 114.65 6.67%
Random Forest Regressor RobustScaled features 0.9134 210.33 19.25%
XGBoost Regressor RobustScaled features 0.8789 248.81 21.96%
Hybrid SARIMAX + RF Residuals 🏆 Residual Coupling 0.9254 98.11 5.12%
💡
The Hybrid Model outperforms all individual models. By coupling SARIMAX (capturing seasonal time trends) with a Random Forest model trained on the residuals (explaining non-linear climatic fluctuations), we achieve an R² of 0.9254 and reduce the mean absolute error to 5.12%.
Analytical Section 11
Future Forecasting Values (Original scale, Mt)
Out-of-sample predictions for the next 6 seasons (mid-2024 to mid-2027) using SARIMAX with Exponentially-Smoothed Exogenous Forecasts
Season Year Forecasted Production 95% CI Range
Maha 2024/2025 1,942.53 [1,805.10, 2,090.41]
Yala 2025 2,003.28 [1,822.14, 2,202.61]
Maha 2025/2026 2,105.59 [1,889.30, 2,346.70]
Yala 2026 2,230.89 [1,955.10, 2,545.92]
Maha 2026/2027 2,341.22 [2,014.20, 2,721.43]
Yala 2027 2,488.10 [2,102.50, 2,944.51]
Forecast Insights

The forecast reveals a stable upward trend in production, expecting Sri Lanka to surpass 2,400 thousand Metric Tons by Yala 2027 under normal weather assumptions. The predicted exogenous variables prevent the "flatline" projection error present in the original model, resulting in a highly realistic confidence interval (shaded region).

Architecture

Forecasting Pipeline Blueprint

A modular, leakage-free processing architecture combining statistical forecasting and non-linear regression.

Input
Raw Excel
149 records
→
Indexing
PeriodIndex
1950Q2-2024Q2
→
Split
Temporal Split
Train < 2018Q4
→
Scale
RobustScaler
GDP/Inflation/Temp
→
Forecast
Exog HW
Rain & Harvest
→
Linear Model
SARIMAX
order(1,1,1)(1,1,0,2)
→
Residual Model
RF Regressor
Hybrid Coupling
# Hybrid Coupling Pipeline Blueprint
from statsmodels.tsa.statespace.sarimax import SARIMAX
from sklearn.ensemble import RandomForestRegressor

# 1. Fit SARIMAX on Log-scaled Production
sarimax_fit = SARIMAX(y_train_log, exog=exog_train, order=(1,1,1), seasonal_order=(1,1,0,2)).fit(disp=False)
sarimax_fitted_train = np.exp(sarimax_fit.fittedvalues)

# 2. Extract residuals
train_residuals = train_df[TARGET] - sarimax_fitted_train

# 3. Train RF Regressor on residuals
rf_residual_model = RandomForestRegressor(n_estimators=100, random_state=42)
rf_residual_model.fit(X_train_preped, train_residuals)

# 4. Predict and combine
sarimax_test_pred = np.exp(sarimax_fit.forecast(steps=len(test_df), exog=exog_test))
rf_pred_residuals = rf_residual_model.predict(X_test_preped)
hybrid_pred = sarimax_test_pred + rf_pred_residuals
Key Takeaways

Critical Analytics & Findings

Evidence-backed insights that shape the prediction architecture.

🌾
Harvested Acres Dominates Yield
Spearman r = +0.904. Sown-to-Harvested ratio represents the true yield driver; a sudden fall indicates severe monsoonal flooding or drought.
Spearman Rank Test
🌧️
Maha Northeast Monsoon Advantage
Paddy crops in Maha (Northeast monsoon) yield nearly double (Mean 1,488.8 Mt) of Yala (Mean 857.8 Mt).
Seasonal Gap s=2
📉
Stationarity Requires Differencing
ADF test validates log(Production) is non-stationary (p=0.34) but stationary after first differencing (p < 0.0001, d=1).
ADF Stationarity Test
🛡️
Robust Scaler Handles Temperature Outliers
Outliers detected in GDP, Inflation, and Temperature (23 outliers) are scaled using RobustScaler, preserving time order.
No dropped rows
Work Together

Open for Agricultural Analytics

Available for agricultural forecasting, time series consulting, and ML projects on Kaggle and GitHub.

CTA: Ali Naderi

📊
Kaggle
kaggle.com/alinaderi1
🐙
GitHub
github.com/AliNaderiii
📧
Linkedin
linkedin.com/alinaderi-data-scientist