What happened in March?

A lot of people have been asking, hypothesizing, and, frankly, guessing about what effect a global pandemic and quarantine coupled with unprecedented levels of governmental and community response will have on child welfare measures. Anyone who claims certainty right now is selling something. We really don’t know.

We do have a few new data points, however. Florida DCF released its dashboard numbers last week, and they show a reduction in intakes. It’s not nearly as large as I expected. You can get different numbers depending on how you count, so there’s plenty of room for salesmanship, but I would say anywhere from 10-17% down is defensible. A lot of other measures were affected way more. Many were not affected at all.

I created a dashboard to look at all of this. I made it for COVID-19 but you can also review other events in Florida child welfare history. Below are the things I noticed in the data.

Intakes were down

I’m going to use investigation intakes as the measure for this discussion, mostly because DCF has a dashboard on those. Intakes, as far as I understand DCF’s documentation, are completed calls to the hotline that are either accepted for investigation or screened out. I don’t think it includes abandoned calls.

First, let’s look at the actual numbers. Did the floor fall out of intakes in March? Not at all. They were in the middle of the decade’s high and low. Back in January 2010, we got 8,000 fewer calls per month and removed a lot more kids.

Florida statewide investigation intakes. Data source: DCF Dashboards.

It’s hard to read charts with all those ups and downs. To smooth things out, let’s take the year-over-year change in March of each year instead. What you see below is that intakes were down about 10% from March 2019 to March 2020. But, they were also down 8% from March 2018 to March 2019 after being up for five prior years. You could call that a 2-point or a 25% decrease from 8 to 10, but it doesn’t really look like that much to me. It could be part of an ongoing trend that started when Governor DeSantis took office. Again, the number was somewhere in the middle of the decade’s high and low.

Florida abuse investigation intakes for March of each year above, with percent change from previous year below.

Maybe March is just weird. Maybe the previous administration did a huge child abuse awareness campaign in March that increased calls. The next chart looks at all the months at once. It’s a little spaghetti-esque, but you can see the general trends: January and February were pretty normal, but March is down over the last two years. It’s not lower than the summer months, but it’s definitely been dropping in the last two years.

Florida abuse intakes year to year with one line per month.

We need a way to compare months to each other over different years. We can do that using seasonal decomposition. I’ve used this before and always get a lot of questions about it. So, as an example, if you are in the business of selling sunscreen, your sales will go up and down during the year because of the weather (these are called “seasonal factors”). Those seasonal factors are pretty stable over time — summer sales will be more than winter sales every single year. But your sales also change over time due to other forces like the economy or how people feel about tanning (“trends”). Those effects are not the same year to year. To really determine if you had a good month in sales compared to the past, you would want to remove the seasonal effects and the trends so that you’re comparing what is left over (sometimes called the “noise” or “error” value). So let’s do that for abuse calls.

Below are the seasonal factors for statewide intakes. These are the predictable ups and downs you see year to year. The colors represent the seasons to make it prettier. You can see that abuse investigation intakes go up about 12% in April, then down about 10-11% in the summer. That corresponds to those very regular ups and down in the first chart above. There are still around 20,000 abuse calls per month in the Summer — it’s just 22% fewer than in the Spring. Just like in the sunscreen example, we will adjust the numbers to account for these normal ups and downs that are caused by the seasons.

Florida abuse intakes seasonal factors for 2010-2020.

Next we have to filter out the bigger trends by taking a moving average. (I’m using a +/- 6 month median window here.) You can see that abuse calls have remained mostly flat. There was a slight rise at the beginning of 2016 and then again in June 2017. Since January 2019 (a new administration), we’ve seen a slightly decrease. The ends of trend lines are always bumpy, but especially when you have a giant worldwide pandemic sitting right off the edge. As we get more data, this line will get smoother.

Florida abuse intakes trend line. Uses a 13-month moving median.

Now for the magic: if you take the original DCF values and remove both the seasonal effects and the trend line, you get a flat line with easily comparable ups and downs. I turned it into a bar chart to make it easier to read below. I also added two horizontal lines to mark four standard deviations from the mean. (Thanks to anomaly.io for sharing their work.) Anything higher or lower than that is probably an anomaly.

So, what does the graph show? Intakes were probably about 16% below the decade’s seasonal average in March. And, that was very, very rare.

Seasonal decomposition for Florida investigation intakes. The horizontal lines in the lower chart indicate 4 standard deviations from the mean. Anything crossing those lines is likely an anomaly.

There is one other similar dip in intakes. It occured in September 2017 during Hurricane Irma when intakes were 15% lower than expected. There are two smaller dips in June 2018 and June 2019 that I have no explanation for. The highest spike in the other direction was March 2011 (+13%) right after the Barahona case hit the news.

What does all this tell us? That intake numbers dropped in March: 10% over last year or 16% over a typical March in the last decade. Those are hurricane level numbers.

What doesn’t it tell us? Whether those lost calls would have resulted in removals or whether they were low-risk calls that easily got pushed aside when other things became more pressing. There were still 25,000 intakes done. Investigations take about 60 days, so we won’t know more until later. Even then the picture will be murky because we won’t know what wasn’t called in or how to account for a workforce that can’t do in-home inspections except in urgent cases.

What else don’t we know? We don’t know what April will look like. I’ve heard people predict that cases will go up, which makes some intuitive sense. But the data does not show that actually happens after hurricanes, and we don’t know whether this downturn will be a passing moment or a new normal that changes the numbers forever.

What else changed?

Intakes have been getting all the press, but here are some other statewide child welfare measures that are also worth looking at.

Verifications jumped up 12%. You don’t even need math — you can see this in the DCF data. Verifications had been higher than expected for a couple of months, but this may by the March 2020 numbers making the rest of 2020 look stronger than is real. For comparison, verifications went down significantly (-12%) the month of Hurricane Irma. There was no rebound, which is slight evidence that maybe they weren’t that bad.

Seasonal decomposition for Florida investigation verifications.

Removals were pretty normal. They were down, but not by much more than a ton of other months that didn’t have a pandemic in them. For comparison, removals were down 20% during Hurricane Irma and never really bounced back after that. Again, that might be evidence that those were lower-risk removals. The only extraordinary spike in removals was in March 2011 after the Barahona case. I wonder if any of those were removals from adoptive parents?

Seasonal decomposition for Florida removals.

Exits from care were low but normal. Exits were down 11%. For Hurricane Irma, exits went down 20% and rebounded over 4 months.

Seasonal decomposition for Florida exits from care.

Reunifications were very normal. It’s interesting that there were lots of times in the last decade when reunifications dipped low, but this wasn’t one of them. I can’t explain that spike in 2010.

Seasonal decomposition for Florida reunifications.

Guardianships were down 34%. You can see this right in the data, too. During Hurricane Irma, guardianships went down 12%, so this really is a notable decline. That other giant dip is in February 2015, after Phoebe Jonchuck was found dead.

Seasonal decomposition for Florida exits to guardianship.

Adoptions were normal, maybe… I really like this one, because it shows how tricky this stuff is. First, adoptions are very seasonal — tons get done in June and November of each year. Also, when hurricanes and other events have suppressed adoptions in the past, they have shown large spikes a few months later to make up for the backlog.

We expect February to be 9.4% down from the year average. So it is unusual to see in February 2020 there was a giant spike (+64%) in adoptions before the event we expect to dampen the numbers. The increase was driven largely by the Central Region cranking out a lot of adoptions in February. If they saw a lockdown coming and rushed to finish pending adoptions, then that is an effect. If the spike happened because a few large sibling groups get adopted, then that is a coronavirus coincidence. Notice that March 2020 is registering in the numbers as normal, but looks very low. I expect these numbers to level out when we get a few more months of data to determine the wider trends.

Seasonal decomposition for Florida exits to adoption.

Everything else? The tableau dashboard lets you filter by region, CBC, circuit, or county. In addition to the measures above, it also lets you look at major maltreatment categories like substance abuse, domestic violence, physical abuse, sexual abuse, and inadequate supervision. I’ll update it next month when the new numbers come out.

Until then, have fun and stay safe!


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *