Defensive programming

Choose your language:  

What is defensive programming?

Defensive programming involves building checks and safeguards into your code. These might catch problems such as incorrect data formats, invalid parameter combinations, unexpected data ranges, or wrong data structures. It’s not the same as testing, but it complements it: defensive programming adds safeguards inside your code, while testing checks that those safeguards and other features work as intended.

It is a proactive way of writing code that anticipates potential failures, operating on the principle that “anything that can go wrong will go wrong”. This approach is especially useful in research code that may be reused by other people (or by you in a few years time!). It helps prevent subtle, silent mistakes that can creep in through hidden assumptions about how data should look, which settings are allowed together, or what ranges are valid.

By adding these checks early, you make your code more robust and its behaviour more predictable. This can save time later by reducing debugging effort and avoiding the need to redo analyses because of errors that were only discovered after the fact.

Example: calculating waiting times

We will return to the calculate_wait_times() function from our waiting times case study.

As a reminder this was our original code for the calculate_wait_times() function.

def calculate_wait_times(df):
    """
    Add arrival/service datetimes and waiting time in minutes.

    Parameters
    ----------
    df : pandas.DataFrame
        Patient-level data containing `ARRIVAL_DATE`, `ARRIVAL_TIME`,
        `SERVICE_DATE`, and `SERVICE_TIME` columns.

    Returns
    -------
    pandas.DataFrame
        Copy of the input DataFrame with additional columns:
        `arrival_datetime`, `service_datetime`, and `waittime`.
    """
    df = df.copy()

    # Combine date and time columns into datetime columns
    for prefix in ("ARRIVAL", "SERVICE"):
        df[f"{prefix.lower()}_datetime"] = pd.to_datetime(
            df[f"{prefix}_DATE"].astype(str) +
            " " +
            df[f"{prefix}_TIME"].astype(str).str.zfill(4),
            format="%Y-%m-%d %H%M"
        )

    # Waiting time in minutes
    df["waittime"] = (
        df["service_datetime"] - df["arrival_datetime"]
    ) / pd.Timedelta(minutes=1)

    return df

#' Add arrival/service datetimes and waiting time in minutes.
#'
#' @param df Data frame with patient-level data containing `ARRIVAL_DATE`, 
#'   `ARRIVAL_TIME`, `SERVICE_DATE`, and `SERVICE_TIME` columns.
#'
#' @return A copy of the input data frame with additional columns:
#'   `arrival_datetime`, `service_datetime`, and `waittime`.
#'
#' @export
calculate_wait_times <- function(df) {
  df <- df |>
    dplyr::mutate(
      arrival_datetime = lubridate::ymd_hm(
        paste(
          as.character(ARRIVAL_DATE),
          sprintf("%04d", as.integer(ARRIVAL_TIME))
        )
      ),
      service_datetime = lubridate::ymd_hm(
        paste(
          as.character(SERVICE_DATE),
          sprintf("%04d", as.integer(SERVICE_TIME))
        )
      )
    )

  if (any(is.na(df$arrival_datetime) | is.na(df$service_datetime))) {
    stop(
      "Failed to parse arrival or service datetimes; ",
      "check for missing or invalid dates/times."
    )
  }

  df <- df |>
    dplyr::mutate(
      waittime = as.numeric(
        difftime(service_datetime, arrival_datetime, units = "mins")
      )
    )

  df
}

1. Protect your code from invalid and edge case inputs

Guard against invalid inputs

Suggested changes:

• Raise an error if the df parameter passed to the function is not of type dataframe.

• Raise an error if the df parameter does not have one or more of the expected columns within it.

These checks use a hard stop approach. They prevent the function from running when the input is wrong, raising an error with a clear error message so the user can understand what went wrong and how to fix it.

Handle empty data as an edge case

Suggested changes:

• If df is the edge case that does not contain any data (rows) warn the user and return an empty dataframe instead of attempting to perform a calculation (return early).

In this situation, the input is structurally valid but represents an edge case, such as a day with no patients. You could treat this as an error, but if empty data is a legitimate case, it is often better to handle it gracefully. Returning early with an empty DataFrame (and a warning) avoids runtime errors while still keeping the function’s contract: it returns a DataFrame with the expected columns and types.

Additional defensive improvements

Suggested changes:

• Document the possible errors in the function docstring.

• Make error messages as clear and user-friendly as you can.

These extra steps help future users (including you in the future) to understand how the function behaves when something goes wrong. Good documentation and messages make it easier to diagnose problems quickly.

Implementing these defensive programming changes

The first two issues are dealt with through structured exception handling. A TypeError is raised if anything other than a dataframe is passed to the function. A ValueError is raised if the columns (values) of the dataframe are not as expected.

We handle the edge case of an empty dataframe through the combination of an if statement, the python warnings module, and an early return.

You can hover over the code to see the changes highlighted in bold.

import pandas as pd
import warnings

def calculate_wait_times(df):
    """
    Add arrival/service datetimes and waiting time in minutes.

    Parameters
    ----------
    df : pandas.DataFrame
        Patient-level data containing `ARRIVAL_DATE`, `ARRIVAL_TIME`,
        `SERVICE_DATE`, and `SERVICE_TIME` columns.

    Returns
    -------
    pandas.DataFrame
        Copy of the input DataFrame with additional columns:
        `arrival_datetime`, `service_datetime`, and `waittime`.

    Raises
    ------
    TypeError
        If `df` is not a pandas DataFrame.
    ValueError
        If required columns (`ARRIVAL_DATE`, `ARRIVAL_TIME`,
        `SERVICE_DATE`, `SERVICE_TIME`) are missing from the DataFrame.

    Warns
    -----
    UserWarning
        If the input DataFrame is empty.
    """
    # Check `df` is a dataframe. Hard fail using exception.
    if not isinstance(df, pd.DataFrame):
        raise TypeError(f"Expected pandas DataFrame, got {type(df).__name__}")

    # Check required columns are provided. Hard fail using exception.
    required_cols = ["ARRIVAL_DATE", "ARRIVAL_TIME",
                     "SERVICE_DATE", "SERVICE_TIME"]
    missing_cols = [col for col in required_cols if col not in df.columns]
    if missing_cols:
        raise ValueError(f"Missing required columns: {missing_cols}")

    # Check if dataframe is empty. Graceful fail return early + raise warning
    if df.empty:
        df = df.copy()
        warnings.warn("Input DataFrame is empty; returning empty result with expected columns")
        df["arrival_datetime"] = pd.Series(dtype='datetime64[ns]')
        df["service_datetime"] = pd.Series(dtype='datetime64[ns]')
        df["waittime"] = pd.Series(dtype='float64')
        return df

    df = df.copy()

    # Combine date and time columns into datetime columns
    for prefix in ("ARRIVAL", "SERVICE"):
        df[f"{prefix.lower()}_datetime"] = pd.to_datetime(
            df[f"{prefix}_DATE"].astype(str) +
            " " +
            df[f"{prefix}_TIME"].astype(str).str.zfill(4),
            format="%Y-%m-%d %H%M"
        )

    # Waiting time in minutes
    df["waittime"] = (
        df["service_datetime"] - df["arrival_datetime"]
    ) / pd.Timedelta(minutes=1)

    return df

2. Example use cases

Passing the wrong data type

Hypothetically let’s take a scenario where the user’s preprocessing of the data does not match what the function expects. Here the user has the data held in a Python list as an intermediate processing step and passes it to the function before converting to a dataframe. The code terminates immediately and we get a clear error message.

patient_data = [
    {'ARRIVAL_DATE': '2026-01-15', 'ARRIVAL_TIME': '0900'},
    {'ARRIVAL_DATE': '2026-01-15', 'ARRIVAL_TIME': '1030'}
]

result = calculate_wait_times(patient_data)

TypeError: Expected pandas DataFrame, got list

Missing columns

Hypothetically let’s take a scenario where a user is missing the “SERVICE” columns in their table of data. The code now elegantly handles that error and reports back to the user. Note that when an exception is raised the code will terminate.

# User forgot to include SERVICE columns
incomplete_df = pd.DataFrame({
    'ARRIVAL_DATE': ['2026-01-15', '2026-01-15'],
    'ARRIVAL_TIME': ['0900', '1030']
})

result = calculate_wait_times(incomplete_df)

ValueError: Missing required columns: ['SERVICE_DATE', 'SERVICE_TIME']

The edge case

When the dataframe is empty a warning is raised, an empty dataframe is returned and the code continues to run:

# User filters data but no records match
empty_df = pd.DataFrame({
    'ARRIVAL_DATE': [],
    'ARRIVAL_TIME': [],
    'SERVICE_DATE': [],
    'SERVICE_TIME': []
})

result = calculate_wait_times(empty_df)

<string>:44: UserWarning: Input DataFrame is empty; returning empty result with expected columns

The first two issues are dealt with using the R stop() function. This is called if anything other than a dataframe is passed to the function or raised if the columns (values) of the dataframe are not as expected.

We handle the edge case of an empty dataframe through the combination of an if statement, the warning() function, and an early return.

#' Add arrival/service datetimes and waiting time in minutes.
#'
#' @param df Data frame with patient-level data containing `ARRIVAL_DATE`, 
#'   `ARRIVAL_TIME`, `SERVICE_DATE`, and `SERVICE_TIME` columns.
#'
#' @return A copy of the input data frame with additional columns:
#'   `arrival_datetime`, `service_datetime`, and `waittime`.
#'
#' @section Errors:
#' The function will stop with an error if:
#' \itemize{
#'   \item `df` is not a data frame
#'   \item Required columns (`ARRIVAL_DATE`, `ARRIVAL_TIME`,
#'         `SERVICE_DATE`, `SERVICE_TIME`) are missing
#' }
#'
#' @section Warnings:
#' A warning is issued if the input data frame is empty (has no rows).
#'
#' @export
calculate_wait_times <- function(df) {

  # Check `df` is a data frame. Hard fail using stop().
  if (!is.data.frame(df)) {
    stop(sprintf("Expected data frame, got %s", class(df)[1]))
  }
  
  # Check required columns are provided. Hard fail using stop().
  required_cols <- c("ARRIVAL_DATE", "ARRIVAL_TIME", "SERVICE_DATE", "SERVICE_TIME")
  missing_cols <- setdiff(required_cols, names(df))
  if (length(missing_cols) > 0) {
    stop(sprintf("Missing required columns: %s", paste(missing_cols, collapse = ", ")))
  }
  
  # Check if data frame is empty. Graceful fail: return early + raise warning
  if (nrow(df) == 0) {
    warning("Input data frame is empty; returning empty result with expected columns")
    df$arrival_datetime <- as.POSIXct(character(0))
    df$service_datetime <- as.POSIXct(character(0))
    df$waittime <- numeric(0)
    return(df)
  }

  df <- df |>
    dplyr::mutate(
      arrival_datetime = lubridate::ymd_hm(
        paste(
          as.character(ARRIVAL_DATE),
          sprintf("%04d", as.integer(ARRIVAL_TIME))
        )
      ),
      service_datetime = lubridate::ymd_hm(
        paste(
          as.character(SERVICE_DATE),
          sprintf("%04d", as.integer(SERVICE_TIME))
        )
      )
    )

  if (any(is.na(df$arrival_datetime) | is.na(df$service_datetime))) {
    stop(
      "Failed to parse arrival or service datetimes; ",
      "check for missing or invalid dates/times."
    )
  }

  df <- df |>
    dplyr::mutate(
      waittime = as.numeric(
        difftime(service_datetime, arrival_datetime, units = "mins")
      )
    )

  df
}

2. Example use cases

Passing the wrong data type

Hypothetically let’s take a scenario where the user’s preprocessing of the data does not match what the function expects. Here the user has the data held in an R list as an intermediate processing step and passes it to the function before converting to a data frame. The code terminates immediately and we get a clear error message.

patient_data <- list(
  ARRIVAL_DATE = c('2026-01-15', '2026-01-15'),
  ARRIVAL_TIME = c('0900', '1030'),
  SERVICE_DATE = c('2026-01-15', '2026-01-15'),
  SERVICE_TIME = c('0930', '1100')
)

result <- calculate_wait_times(patient_data)

Error in calculate_wait_times(patient_data): Expected data frame, got list

Missing columns

Now let’s take a scenario where a user is missing the “SERVICE” columns in their table of data. The code now elegantly handles that error and reports back to the user. Note that when stop() is called the code will terminate.

incomplete_df <- data.frame(
  ARRIVAL_DATE = c('2026-01-15', '2026-01-15'),
  ARRIVAL_TIME = c('0900', '1030')
)

result <- calculate_wait_times(incomplete_df)

Error in calculate_wait_times(incomplete_df): Missing required columns: SERVICE_DATE, SERVICE_TIME

The edge case

When the dataframe is empty a warning is raised, an empty dataframe is returned and the code continues to run:

empty_df <- data.frame(
  ARRIVAL_DATE = character(0),
  ARRIVAL_TIME = character(0),
  SERVICE_DATE = character(0),
  SERVICE_TIME = character(0)
)

result <- calculate_wait_times(empty_df)

Warning in calculate_wait_times(empty_df): Input data frame is empty; returning
empty result with expected columns

3. Write unit tests to check that the defensive code works as expected.

Once you’ve added defensive programming safeguards to your code, it’s essential to verify they work correctly. This is where the testing approaches we’ve covered earlier come into play. Note we are testing for “negative” behaviour here - i.e., that the code fails as we expect.

Here are some examples to test the defensive additions to calculate_wait_times()

• Test that passing a different data structure than a DataFrame raises the appropriate error and message.
• Test that passing a DataFrame missing one or more required columns raises the appropriate error and message.
• Test that an empty DataFrame returns the correct structure with arrival_datetime, service_datetime, and waittime columns
• If implemented, test that an empty DataFrame with valid column triggers a warning.

4. What else could we do defensively?

The example we provide here only scratches the surface of defensive programming. We have introduced guardrails for our inputs, but the data within them could also be garbage. For example, the function expects date time data to be in a specific format. Our df with a different format might fail in unexpected ways, potentially causing a runtime error, or even worse silently!

There is only so much time you have available to write defensive code, and deciding what to guard against is critical. This is why testing your code is so important. If, for example, you test your code and find it fails silently i.e. no error message, then you should strongly consider introducing a defensive programming routine to prevent that from happening in the future.

Remember: the goal is not to defensively program against every conceivable scenario, but to protect against the most likely errors and those with the most serious consequences for your research results.