Coding Standards#
Learning to code is hard. Learning to code to a level suitable for industry, research, and publication is harder.
When you work in a health data science job you will quickly find that sloppy, inconsistent or confused code isn’t acceptable to your fellow data scientists. Unclean code that does not follow a common style leads to (more) mistakes, difficulties in others understanding your aim(s), and hampers the ability for others (and yourself) to maintain the code base longer term.
Python is no exception. I encourage you to look at and use Python’s PEP8 guidelines coding standards. Where I can I try to follow PEP8 standards with my own code. I’m not always successful. If you see any code in this book that you think falls short of a good standard then do let me know and I will revise it!
To help you on your clean code journal I’ll summarise a few of the quick wins here. If you follow these I guarantee it will help clean up your code.
Quick Wins for coding standards#
As noted I strongly recommend checking out PEP8 in detail. However, here are a few quick wins to get you started:
Variable naming#
We want to be able to store data and results of calculations in ways we can re-use. For this we define variables names. In python variables names can only contain letters, numbers, and underscores.
In python we tend to prefer using variables names that are all lower case with under scores.Here is a simple pythonic versus non-pythonic style example:
# preferred style
hospital_id = 1
# not pythonic
hospitalID = 1
# another non-pythonic style
Hospital_id = 1
Sometimes python modules and packages will contain module level scope variables that you wish to treat as constants (as python is dynamically typed there is no real constant primitive). In these cases it is usual to use uppercase with underscores. For example:
DEFAULT_N_PATIENTS = 172
DEFAULT_N_WARDS = 13
Give variables meaningful names#
Variable names should be descriptive, without being too long. For example mc_wheels is better than just wheels, and number_of_wheels_on_a_motorycle.
That’s a bit abstract. Let’s take a look at a more practical example below. Scan over the code listing quickly. What is its purpose?
i = 30_000.
r = 0.2
j = i * (1 - r)
print(i, j)
30000.0 24000.0
Here is the same code again. This time with more meaningful variable names
salary = 30_000.
tax_rate = 0.2
salary_after_tax = salary * (1 - tax_rate)
print(salary, salary_after_tax)
30000.0 24000.0
Which of the two code listings do you think is easier to read and maintain?
Line length#
In python we usually keep line length to 79 or 80 characters. That is a line that exceeds this length breaks PEP8 coding standards. More concretely - it becomes harder to read the code! This is partly due to (old) monitor size, but also coders usually have more than one window open at a time. Long unbroken lines of code can be hard to read.
There’s no real excuse for having extremely long lines of code. Python syntax and all major IDEs makes it easy to split lines. In Jupyter, for example, make sure to head over to advanced settings and add in a ruler for code cells at either 79 or 80 characters. Here is a simple example where the backslash character is used to break up a string over multiple lines.
# splitting a string over multiple lines with the backslash
msg = 'Invalid parameter selections for hospital_id, ward_id and ' \
'confidence_int. Please select values with range provided in the main ' \
'manual.'
print(msg)
Invalid parameter selections for hospital_id, ward_id and confidence_int. Please select values with range provided in the main manual.
In the data science libraries of python you will find that functions and classes tend to have a large number of mandatory and optional parameters. To keep your code readable you will need to split your code over multiple lines. For example, here is a dummy function that takes four parameters:
def rolling_forecast_origin(train, min_train_size, horizon, step):
'''dummy function with lots of parameters'''
return None
y_train = []
min_train_size = 24
horizon = 6
step = 2
# call the function
cv = rolling_forecast_origin(train=y_train, min_train_size=min_train_size,
horizon=horizon, step=step)
Alternatively you can put each parameter on a seperate line:
# alternative call style
cv = rolling_forecast_origin(train=y_train,
min_train_size=min_train_size,
horizon=horizon,
step=step)
Docstrings#
If you create a python function, class, or module then you should provide a docstring to go with it. You can read more about docstrings in PEP 257. I’ve provided a simple function example below (with the main code ommitted). Note that use of the triple quotes to open and close the docstring. In this case my docstring consists of three sections:
A short description of the singular purpose of the function
A description of mandatory and optional parameters (and default values if applicable)
Details of the type of variable(s) returned when execution is complete.
def multiple_replications(rc_period=1440, warm_up=0,
n_reps=5):
'''
Perform multiple replications of a computer simulation model
of a hospital ward. Returns results of each replication in tabular
format.
Params:
------
rc_period: float, optional (default=1440)
results collection period.
the number of minutes to run the model beyond warm up
to collect results
warm_up: float, optional (default=0)
initial transient period. no results are collected in this period
n_reps: int, optional (default=5)
Number of independent replications to run.
Returns:
--------
pandas.DataFrame
'''
pass
Docstrings can vary in length depending on the complexity of the code. For example, from my forecast_tools package I include the following docstring with a function called auto_naive . This includes additional sections on:
Raises - a list of exceptions that can occur when called
See also - a list of related classes and functions
Examples - pythonic code to test the function
def auto_naive(y_train, horizon=1, seasonal_period=1, min_train_size='auto',
method='cv', step=1, window_size='auto', metric='mae'):
'''Automatic selection of the 'best' naive benchmark on a 'single' series
The selection process uses out-of-sample cv performance.
By default auto_naive uses cross validation to estimate the mean
point forecast peformance of all naive methods. It selects the method
with the lowest point forecast metric on average.
If there is limited data for training a basic holdout sample could be
used.
Dev note: the plan is to update this to work with multiple series.
It would be best to use MASE for multiple series comparison.
Parameters:
----------
y_train: array-like
training data. typically in a pandas.Series, pandas.DataFrame
or numpy.ndarray format.
horizon: int, optional (default=1)
Forecast horizon.
seasonal_period: int, optional (default=1)
Frequency of the data. E.g. 7 for weekly pattern, 12 for monthly
365 for daily.
min_train_size: int or str, optional (default='auto')
The size of the initial training set (if method=='ro' or 'sw').
If 'auto' then then min_train_size is set to len(y_train) // 3
If main_train_size='auto' and method='holdout' then
min_train_size = len(y_train) - horizon.
method: str, optional (default='cv')
out of sample selection method.
'ro' - rolling forecast origin
'sw' - sliding window
'cv' - scores from both ro and sw
'holdout' - single train/test split
Methods'ro' and 'sw' are similar, however, sw has a fixed
window_size and drops older data from training.
step: int, optional (default=1)
The stride/step of the cross-validation. I.e. the number
of observations to move forward between folds.
window_size: str or int, optional (default='auto')
The window_size if using sliding window cross validation
When 'auto' and method='sw' then
window_size=len(y_train) // 3
metric: str, optional (default='mae')
The metric to measure out of sample accuracy.
Options: mase, mae, mape, smape, mse, rmse, me.
Returns:
--------
dict
'model': baseline.Forecast
f'{metric}': float
Contains the model and its CV performance.
Raises:
-------
ValueError
For invalid method, metric, window_size parameters
See Also:
--------
forecast_tools.baseline.Naive1
forecast_tools.baseline.SNaive
forecast_tools.baseline.Drift
forecast_tools.baseline.Average
forecast_tools.baseline.EnsembleNaive
forecast_tools.baseline.baseline_estimators
forecast_tools.model_selection.rolling_forecast_origin
forecast_tools.model_selection.sliding_window
forecast_tools.model_selection.mase_cross_validation_score
forecast_tools.metrics.mean_absolute_scaled_error
Examples:
---------
Measuring MAE and taking the best method using both
rolling origin and sliding window cross validation
of a 56 day forecast.
```
>>> from forecast_tools.datasets import load_emergency_dept
>>> y_train = load_emergency_dept
>>> best = auto_naive(y_train, seasonal_period=7, horizon=56)
>>> best
{'model': Average(), 'mae': 19.63791579700355}
```
Take a step of 7 days between cv folds.
```
>>> from forecast_tools.datasets import load_emergency_dept
>>> y_train = load_emergency_dept
>>> best = auto_naive(y_train, seasonal_period=7, horizon=56,
... step=7)
>>> best
{'model': Average(), 'mae': 19.675635558539383}
```
'''
pass
Linting python code#
If you are new to coding you might be delighted to know that software exists to help you write cleaner code and keep to standards. These are called code linters.
There are a number of linters you can choose from. Here I direct to flake8. I’ve always found this helpful.
To use flake8 with a Jupyter notebook requires another package nbqa. This can be installed from pip.
For example to run the linter with this particular notebook I would run the following in the terminal:
nbqa flake8 content/001_setup/prereq/05_pep8.ipynb
Here’s a short extract of what it returned.
content/001_setup/prereq/05_pep8.ipynb:cell_3:3:80: E501 line too long (80 > 79 characters)
content/001_setup/prereq/05_pep8.ipynb:cell_4:5:1: E305 expected 2 blank lines after class or function definition, found 1
content/001_setup/prereq/05_pep8.ipynb:cell_5:2:44: W291 trailing whitespace
content/001_setup/prereq/05_pep8.ipynb:cell_5:4:46: W291 trailing whitespace
As an example, the first line tells me that the third cell breaks is over 79 characters in length. I can go and fix that, save the file and rerun the linter. (In this case its not too much of a big deal).