Michael Remington

Using Machine Learning to Group US States by Covid-19 Hospitalization Trends

Michael Remington — Tue, 01 Jun 2021 19:20:20 GMT

Overview

Machine learning algorithms can provide unique insights from Covid-19 data. In this article I'll use a clustering algorithm to group US states based on trends in Covid-19 hospitalizations and other metrics. Some groupings are surprising and invite further investigation. Why would a state have a different trend than its neighbors? Why would some distant states have similar trends? These results may hint at patterns of interstate travel during the pandemic.

First we'll explore the hospitalization trend groupings, then we'll add more metrics involving cases and deaths. Lastly, we'll dive into how the project works.

Data Preprocessing

All data was scaled from 0-1 to account for population differences. We are looking at similar hospitalization trends and timing, not similar numbers hospitalized. Without scaling the groups are predictable and uninteresting: states with similar populations are grouped.

All states before scaling. States with higher populations have taller peaks. Note: values dip below 0 because they have been centered by Scikit-Learn's PCA implementation.

After scaling, all state trends are equal height.

Results

The OPTICS clustering algorithm produced these groupings. The inputs were Covid-19 hospitalizations from March 2020 to March 2021 for each state.

Group 1

Hospitalizations in these states followed nearly identical trends and peaked within 15 days of each other. This group has 3 bordering states.

Group 2

The geographical outlier is Washington State. Despite this, the hospitalization trajectories show a similar plateau in hospitalizations around the same time.

Group 3

Despite sharing borders with the last group, the clustering algorithm separates Delaware, Massachusetts, and New Hampshire. These states show a narrower and later peak in hospitalizations compared to their neighbors in group 2. Delaware is particularly interesting - it's completely surrounded by states from group 2 but does not share their hospitalization curve.

Group 4

These hospitalization trends are similar but the states are distant. Their peaks occurred within ten days of each other and have similar shapes. There may be a non-obvious commonality that led to similar trends.

Additional Hospitalization Groups

Lowering the minimum states per group from 3 to 2 results in additional groups. Note that AZ, TX, LA, and MS experienced two large peaks in hospitalizations, while DC, NY, NC, and VA experienced just one large peak simultaneously.

Two Peaks

One Peak

All Hospitalization Trend Groups

Adding Cases, Deaths, and Other Metrics

Using my web interface, we'll now add four additional metrics to cluster states by. Precise definitions for these metrics are found in the methods section.

The clustering algorithm returns new groups after taking into account the five metrics above.

The line charts now attempt to represent all five metrics in each line. This is done with dimensionality reduction (more details later).

Group 1

The absence of the Dakotas is notable.

Group 2

Group 3

We again see Washington as a geographical outlier even though its chart shows similar trends in the five metrics.

Group 4

Group 5

These southern states show similar trajectories in the five metrics. They have two major peaks while the previous groups had just one.

All Trend Groups

Additional groups 6 and 7 were created by lowering the minimum states per group from 3 to 2.

Discussion

I'm a data scientist, not an infectious disease expert, but I'll speculate that interstate travel, population density, and related elements may be key factors for these groups.

For example, we can presume that there was more interstate travel between North and South Dakota than between South Dakota and any of group 1 (red in the above map). Other groups like group 3 (green) are less explainable due to geographical outliers.

Methods

All data is from covidtracking.com. The data was scaled from 0-1 as explained earlier.

Data Definitions:

hospitalizedCurrently: Number hospitalized for Covid-19 at any point in time. Not just new admissions.
deathIncrease: New Covid-19 deaths per day. This is smoothed with a one-week average as is common in prominent publications such as the New York Times.
inIcuCurrently: Number in an ICU for Covid-19 at any point in time. Not just new admissions.
percentPostive: The fraction of tests that are positive. I believe this is a better comparison than cases per million because of varying case definitions and testing rates between states.
Case Fatality Rate: Fraction of deaths out of known cases for each state. This is not the lethality of Covid-19, which is instead estimated by the infection fatality rate (IFR).

Reducing Dimensions

If more than one metric is selected then dimensionality reduction is done with PCA. PCA projects the metrics into a lower-dimensional space while attempting to preserve the information that explains the most variance. A potential improvement would involve clustering in all five dimensions and reducing dimensions only for visualization. I implemented a T-SNE function for this purpose.

The OPTICS clustering algorithm was used to generate clusters. The minimum number of datapoints per cluster was 3 unless otherwise stated.

Follow me for more articles like this:

https://twitter.com/mremingtn

Source code

github.com/remingm/covid19-clustering-states

Interactive Covid-19 population immunity estimates

Michael Remington — Sun, 07 Mar 2021 02:02:00 GMT

I made a web app that estimates population immunity for SARS-CoV-19. Try it out at covid.mremington.co.

covid.mremington.co

What follows is an overview of how the estimate works. Feel free to skip this post and just explore the site if you're not interested in the behind-the-scenes.

First, a crash course in population immunity:

Why is population immunity important?

Normal life may safely return when enough of the population has immunity to Covid-19, limiting further spread. This is known as "herd immunity". [1]
Herd immunity may be achieved either through infection and recovery or by vaccination. [2]
Besides protecting the individual, the goal of vaccination is for a population to reach herd immunity safely. [2]
Herd immunity also protects those who are unable to be vaccinated, such as newborns and immunocompromised people, because the disease spread within the population is very limited. [2]

How much of the population needs immunity?

The herd immunity threshold (HIT) is debated among scientists. The commonly accepted herd immunity threshold for SARS-CoV-2 is 60-80%. [3] [1]
A research group at the University of Oxford estimates the threshold at 10% to 60% when accounting for T cell immunity studies. [3]
Another research group calculates the threshold at 10% to 20% when accounting for diversity in population mixing. Some consider this controversial. [3]
Infections, hospitalizations, and related metrics may decline as population immunity rises, even if the HIT is not reached. [1]

References

To estimate population immunity we need to know:
1. The number of vaccinated people
2. The number of people who were infected and recovered
3. The overlap between these groups

For vaccinations my code simply pulls the data daily. You can choose that either the first or second dose be counted as immunity.

Determining recovered infections is more tricky. Cases only represent a fraction of known infections, since not all infected people get tested. Luckily Youyang Gu, one of the top Covid-19 modelers, has presented a simple equation for estimating true infections. To use this equation we need the daily percentage of positive tests and the daily number of new cases.

Many who get vaccinated have already been infected. It's impossible to know the exact overlap, but if we don't estimate it then our immunity estimate will be too high. One of the top modelers assumes 50% of vaccinations go to recovered infections. In my web app that is customizable, with a default of 20%. 28% of the US is estimated to have been infected as of March 7, 2021.

I also used SciPy to find the last peak in hospitalizations and report the estimated population immunity at that time. Note that this is not necessarily the HIT, as seasonality and other factors also affect transmission. My model found that US hospitalizations began to decline from their all-time peak after January 12, 2021 when the estimated population immunity was 22.7%.

Putting it all together, here is the complete web app, covid.mremington.co, as of March 7th, 2021:

covid.mremington.co

The source code is available here.

Lastly, I was greatly inspired by Youyang Gu of covid19-projections.com, who built a similar model to estimate immunity.