Skip to contents

Why add another model?

Bins can beat a straight line because the relationship is curved. That does not establish that categories are the best representation. A useful comparison includes a continuous model that can bend while keeping the original measurements.

A natural cubic spline fits connected smooth curves. Its knots are the locations where those curve pieces join. A knot is not a category boundary; values on either side remain part of a continuous fitted relationship. The curve becomes linear outside its outer boundary knots on the model’s link scale. In a logistic model that means the log-odds scale, not the probability scale. R spline documentation.

In binwise, adding spline_df = 3 includes this comparator under the name spline. Three degrees of freedom is a modest starting amount of flexibility. Larger values allow more complex shapes and can overfit. It is not an automatic choice of the best complexity. The original continuous model remains the linear-term reference. For logistic models its terms are linear in log-odds.

Start a fresh R session, load the package, and run these blocks in order. See the getting-started guide if you need help running code in RStudio.

library(binwise)
data("binwise_smooth", package = "binwise")

1. Create a curved teaching example

We will reuse the existing exposure values and simulated random noise, but change the known expected outcome to a smooth U-shaped relationship. The original dataset remains unchanged.

curved <- binwise_smooth
noise <- curved$outcome - curved$expected_outcome
curved$expected_outcome <- 2 + .008 * (curved$exposure - 50)^2
curved$outcome <- curved$expected_outcome + noise
plot(curved$exposure, curved$outcome, pch = 16, cex = .4, col = "grey65",
     xlab = "Exposure", ylab = "Observed outcome")

The outcome tends to be higher near both ends and lower in the middle. The expected_outcome column is available only because this is a simulation. Do not include that teaching aid as a control when predicting the noisy outcome.

2. Fit the straight line, spline, and bins

quartiles <- bin_diagnose(curved, "exposure", bins = 4, warn = FALSE)
curved_fit <- bin_interact(
  curved, outcome = "outcome", variables = "exposure",
  schemes = list(quartiles = quartiles), spline_df = 3
)
plot(curved_fit, xlab = "Exposure", main = "A curve can explain why bins beat a line")

curved_fit$comparison
#>        model    n parameters  AIC  BIC training_score status delta_AIC
#> 1 continuous 1000          3 6659 6674          6.736     ok       0.0
#> 2     spline 1000          5 5068 5092          3.034     ok   -1591.3
#> 3  quartiles 1000          5 5844 5869          4.473     ok    -814.8

The legend identifies three models. continuous fits a straight line here. spline can follow the smooth change. quartiles fits flat steps that discard variation inside each interval. All models use the same observations.

Shading shows 95% pointwise confidence intervals for each fitted mean. It conditions on the specified model and complexity; it does not account for choosing the preferred degrees of freedom after viewing results. The bands are not prediction intervals for individual observations, and their overlap is not a test of whether two model shapes are equivalent.

Read the plain-language assessment and inspect the spline definition.

cat(paste(curved_fit$assessment, collapse = "\n\n"))
#> All models use the same 1000 observations; 0 rows were excluded for missing or nonfinite inputs. This assessment compares your supplied schemes with the continuous benchmark and does not choose bins for you.
#> 
#> These models have no interaction. Comparisons concern the representation of the focal predictors with additive controls, if supplied.
#> 
#> For the continuous benchmark, moving 'exposure' from 0 to 100 gives a decrease of 0.4014 outcome units. These are end-to-end predicted changes on the displayed grid, not per-unit effects or causal estimates; controls stay fixed.
#> 
#> Natural spline comparator (3 degrees of freedom per focal variable). Compared with continuous, AIC is 1591 lower; BIC is 1581 lower; training RMSE is 3.702 lower. The largest absolute prediction difference on the default grid is 12.76 outcome units (exposure = 0; 107 of 107 grid rows comparable). Decide whether that change matters for your application. This model retains numeric variation within the fitted curves; it has no bin-group counts. Inspect joint support and validate predictions.
#> 
#> Scheme 'quartiles'. Compared with continuous, AIC is 814.8 lower; BIC is 804.9 lower; training RMSE is 2.263 lower. The largest absolute prediction difference on the default grid is 5.197 outcome units (exposure just above 25; 107 of 107 grid rows comparable). Decide whether that change matters for your application. Its bin groups contain 250 to 250 observations; 0 groups are empty and 0 fall below your count screen (n < 20).
#> 
#> The spline is a prespecified flexible continuous alternative, not an automatically tuned model. Beating the linear reference alone does not demonstrate an advantage over the spline.
#> 
#> Scheme 'quartiles' compared with spline: AIC is 776.5 higher; BIC is 776.5 higher; training RMSE is 1.439 higher. These are in-sample comparisons; use bin_validate for held-out prediction.
#> 
#> Lower AIC or BIC indicates a better fit-complexity tradeoff within this common sample and model family. Lower training error describes these fitted data only. These comparisons are not significance tests or validation on new observations.
#> 
#> Binning treats values within a category alike, holding other predictors fixed. Consider it when the boundaries have a defensible meaning and the lost within-bin distinctions are acceptable. Lower AIC alone cannot establish a real threshold. A smooth nonlinear alternative may improve on the linear-term reference; any supplied spline is one such alternative, not every possible continuous model.
#> 
#> Prediction differences refer only to the displayed grid with controls fixed, not the whole sample or every possible covariate combination. Group counts describe rows, not independent units, and do not establish joint support for continuous predictors.
#> 
#> Confidence bands are pointwise and model-based with fixed model settings, including cutpoints and spline complexity. Their width or overlap does not validate the model shape or test differences between models. Bin selection and clustered dependence are not accounted for. No scheme is selected automatically.
curved_fit$spline_details
#> $exposure
#> $exposure$df
#> [1] 3
#> 
#> $exposure$knots
#> [1] 33.33 66.67
#> 
#> $exposure$boundary_knots
#> [1]   0 100
#> 
#> $exposure$training_n
#> [1] 1000

The assessment now includes each binned scheme’s fit comparison with the spline. AIC, BIC, and training error still describe the fitted sample. They do not replace the held-out comparison below. spline_details records interior knots, boundary knots, degrees of freedom, and the number of training observations.

3. Compare held-out predictions directly against the spline

curved_check <- bin_validate(
  curved, outcome = "outcome", variables = "exposure",
  schemes = list(quartiles = list(
    exposure = list(probs = c(0, .25, .5, .75, 1))
  )),
  spline_df = 3, reference = "spline", folds = 5, seed = 12, reps = 199
)
curved_check$scores
#>        model metric score lower upper
#> 1     spline   RMSE 3.051 2.929 3.165
#> 2 continuous   RMSE 6.759 6.496 6.990
#> 3  quartiles   RMSE 4.505 4.312 4.694
#> 4     spline    MAE 2.436 2.329 2.543
#> 5 continuous    MAE 5.592 5.345 5.804
#> 6  quartiles    MAE 3.577 3.414 3.744
plot(curved_check, metric = "RMSE",
     main = "Positive differences mean worse predictions than the spline")

The RMSE values are approximately 6.76 for the straight line, 4.50 for quartile bins, and 3.05 for the spline. Lower is better. The bins improve on the straight line, but the spline does considerably better than the bins. Here, the useful feature is curvature, and categorization loses information about it.

reference = "spline" changes which model is subtracted when computing score differences. It does not change the fits, folds, or observations scored. A negative difference favors a candidate over the spline; a positive one favors the spline. The intervals are paired bootstrap intervals for loss differences with predictions and folds fixed. They do not cover all uncertainty from refitting or model selection. Increase reps for a final analysis and check interval stability.

The spline knots and quantile boundaries are estimated only from training observations in each fold. Held-out values reuse the training definitions. You can inspect the first fold’s stored spline information.

curved_check$fold_details[[1]]$spline
#> $exposure
#> $exposure$df
#> [1] 3
#> 
#> $exposure$knots
#> [1] 33.67 67.23
#> 
#> $exposure$boundary_knots
#> [1]   0 100
#> 
#> $exposure$training_n
#> [1] 800
curved_check$diagnostics
#>             fold      model training_n test_n unavailable messages
#> continuous     1 continuous        800    200           0         
#> spline         1     spline        800    200           0         
#> quartiles      1  quartiles        800    200           0         
#> continuous1    2 continuous        800    200           0         
#> spline1        2     spline        800    200           0         
#> quartiles1     2  quartiles        800    200           0         
#> continuous2    3 continuous        800    200           0         
#> spline2        3     spline        800    200           0         
#> quartiles2     3  quartiles        800    200           0         
#> continuous3    4 continuous        800    200           0         
#> spline3        4     spline        800    200           0         
#> quartiles3     4  quartiles        800    200           0         
#> continuous4    5 continuous        800    200           0         
#> spline4        5     spline        800    200           0         
#> quartiles4     5  quartiles        800    200           0

The spline extends linearly on the link scale beyond its training boundary knots. That defines an extrapolation rule; it does not prove that extrapolation is accurate. Check coverage, support, and the intended prediction population.

4. Do bins still help when there is a real threshold?

The interaction simulation has an outcome whose expected value jumps at zero. Within each side of that boundary, changes in x do not change the expected outcome when z and the control are held fixed. That known rule supplies a reason to test two categories.

data("binwise_interaction", package = "binwise")
threshold_check <- bin_validate(
  binwise_interaction, outcome = "threshold_outcome",
  variables = c("x", "z"), controls = "control",
  schemes = list(threshold = list(x = list(breaks = c(-Inf, 0, Inf)))),
  spline_df = 3, reference = "spline", folds = 5, seed = 12, reps = 199
)
threshold_check$scores
#>        model metric  score  lower  upper
#> 1     spline   RMSE 1.2763 1.2213 1.3261
#> 2 continuous   RMSE 1.3430 1.2909 1.3981
#> 3  threshold   RMSE 1.1981 1.1462 1.2442
#> 4     spline    MAE 1.0215 0.9763 1.0629
#> 5 continuous    MAE 1.0669 1.0217 1.1150
#> 6  threshold    MAE 0.9537 0.9085 0.9936
plot(threshold_check, metric = "RMSE", main = "A real jump can favor threshold categories")

Here, RMSE is approximately 1.34 for the linear interaction, 1.28 for the spline, and 1.20 for the threshold categories. The bins still help against this more flexible comparator. Their advantage follows from this simulated jump; it is not a general recommendation to bin all variables at zero.

With two focal variables, spline_df = 3 applies a spline to both. The default interaction includes all products of those basis terms plus their main effects. This uses 15 focal-predictor terms, plus an intercept and any controls, compared with three terms for a linear interaction. With limited observations or joint support, that can be too complex. Set interaction = FALSE for additive splines when that matches your question. The package records fitting failures rather than silently reducing complexity. Numeric controls remain linear and additive in all candidates.

For a binary outcome, use model = "logit". Predictions are probabilities, and validation reports Brier scores and log loss instead of RMSE and MAE. Lower remains better. The same spline settings and training-only rules apply.

5. Readable boundaries are another candidate, not an automatic decision

R’s pretty() can suggest round boundaries for a descriptive table or plot. They are useful when communication matters. You can already use them through bin_compare() without installing another package.

round_breaks <- pretty(curved$exposure, n = 4)
round_comparison <- bin_compare(
  curved, "exposure", methods = "quantile",
  custom_breaks = list(round_boundaries = round_breaks)
)
round_breaks
#> [1]   0  20  40  60  80 100
round_comparison$bins[, c("scheme", "bin", "lower", "upper", "n")]
#>             scheme bin lower upper   n
#> 1       quantile_4  B1     0    25 250
#> 2       quantile_4  B2    25    50 250
#> 3       quantile_4  B3    50    75 250
#> 4       quantile_4  B4    75   100 250
#> 5 round_boundaries  B1     0    20 200
#> 6 round_boundaries  B2    20    40 200
#> 7 round_boundaries  B3    40    60 200
#> 8 round_boundaries  B4    60    80 200
#> 9 round_boundaries  B5    80   100 200

The requested count is approximate, so inspect the actual boundaries and counts. In this example, round boundaries give five intervals of width 20. These are convenient labels, not known scientific thresholds. R documentation for pretty().

Do not calculate boundaries from the whole dataset and then describe them as training-only. In validation, custom breaks mean fixed, prespecified boundaries. A rule that learns pretty() or clustering boundaries from data must be fitted inside each training fold, currently through a fit_predict callback. Built-in recipes currently learn quantiles; they do not learn clustering or pretty breaks.

How to use this comparison responsibly

Choose a small set of plausible schemes and spline complexities before examining test performance. Searching many alternatives and reporting the winner on the same test observations exaggerates performance. Use nested validation or a fresh test set for that selection process.

A single three-degree-of-freedom spline is not every possible continuous model. Smooth splines also do not automatically preserve a meaningful zero state or model a discontinuity. A zero indicator plus continuous positive intensity may be appropriate for another question. Interpretation and prediction are separate considerations, and none of these comparisons alone identifies a causal effect.

For repeated observations, supply an appropriate grouping column. For temporal prediction, note that these folds fit on all other folds, including later ones; this is not forward-only forecasting. Inspect diagnostics before interpreting scores calculated on only the rows with predictions available from every model.

The companion real-data guide now includes a spline in the air-quality comparison. Open it with vignette("real-data-and-related-work", package = "binwise").