District of Columbia

Rows edited: 1

DC 2021-02-08 recovered: 26986 (was 28986)

Rows edited: 1

DC 2021-02-08 recovered: 26986 (was 28986)

Rows edited: 1 DC 2021-02-12 totalTestEncountersViral: 1154309 (was 1151484) totalTestsPeopleViral: 427581 (was 421753)

Rows edited: 1 DC 2021-02-12 totalTestEncountersViral: 1154309 (was 1151484) totalTestsPeopleViral: 427581 (was 421753)

Alaska: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

California: Never reported negatives directly, but did report in Total tests (people) until April 21, 2020. Removing time series from present to April 22, 2020.

Values Removed: Changes.txt

Washington DC: Never reported negatives directly, and has always reported encounters, removing total time series

Values Removed: Changes.txt

Georgia: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

Kentucky: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

New York: Never reported negatives directly and always reported in encounters, removing total time series

Values Removed: Changes.txt

Ohio: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

Oregon: Never reported negatives directly, but did report in Total tests (people) until December 1, 2020. Removing time series from present to December 2, 2020.

Values Removed: Changes.txt

Texas: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

Virginia: Never reported negatives directly and always reported in encounters, removing total time series

Values Removed: Changes.txt

Washington: Negatives were backfilled with values calculated from total tests (encounters)-confirmed cases in August 2020. Removing total time series.

Values Removed: Changes.txt

Alaska: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

California: Never reported negatives directly, but did report in Total tests (people) until April 21, 2020. Removing time series from present to April 22, 2020.

Values Removed: Changes.txt

Washington DC: Never reported negatives directly, and has always reported encounters, removing total time series

Values Removed: Changes.txt

Georgia: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

Kentucky: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

New York: Never reported negatives directly and always reported in encounters, removing total time series

Values Removed: Changes.txt

Ohio: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

Oregon: Never reported negatives directly, but did report in Total tests (people) until December 1, 2020. Removing time series from present to December 2, 2020.

Values Removed: Changes.txt

Texas: Never reported negatives directly and always reported in specimens, removing total time series

Values Removed: Changes.txt

Virginia: Never reported negatives directly and always reported in encounters, removing total time series

Values Removed: Changes.txt

Washington: Negatives were backfilled with values calculated from total tests (encounters)-confirmed cases in August 2020. Removing total time series.

Values Removed: Changes.txt

I also patched 5/23 and 5/24 for total test encounters the day the timeseries became available to reflect revisions because it looks like we patched with the wrong values.

The drop on 5/23 was due to a review of data https://coronavirus.dc.gov/release/coronavirus-data-may-23-2020

Before

After

Before:

After:

New WS2 tooltip

BEFORE -

AFTER - Corrected historicals by removing values from Positive Cases (PCR).

Edited the tooltip-note and also added a process note.

1. I deleted the value from Worksheet2

2. I deleted the values from States Daily BEFORE

AFTER

Hi @JustOneGuyHere

Please see the CDC Data tracker here https://www.cdc.gov/covid-data-tracker/index.html

Currently 65,735 deaths are reported on the page.

Thanks, censor.

On Sun, May 3, 2020 at 6:35 PM Michal Mart notifications@github.com wrote:

Closed #352 https://github.com/COVID19Tracking/issues/issues/352.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/COVID19Tracking/issues/issues/352#event-3298064864, or unsubscribe https://github.com/notifications/unsubscribe-auth/APOCEYR7QRRQTI7T4VA4XO3RPYLWJANCNFSM4MYLPALA .

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

Also, on 4/7 in DC:

"deathIncrease":-2

I got curious and wrote a tool to find errors like this: https://www.covidcharts.com/data_check.html

Entirely client-side JS, so feel free to re-use any of the code.

@muamichali tells me this is a duplicate of #219

Your tool is great! Thank you!

Per the disclaimer on the CDC page you link, their data includes 100 counties -- these are not statewide totals. CDC is estimating rates based on a sample. Not meant to be used for total counts.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

Thanks, John.

The DC hospitalization numbers are from Mayor Bowser's 4/13 press conference https://www.facebook.com/MayorMurielBowser/videos/846453305836225/

Hi @jdmaresco

The 4/14 negatives number was incorrect. We are going to amend it based on hisorical information DC provides here Looking at screenshots it seems that the state reported a total number of tests and then revised it down the next day.

Before

After

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

Thanks, Patrick!

We are discussing how to handle the probable deaths as its own metric.

For now, we are including public notes about each state where the probable deaths are included and we do include the probable deaths within the overall deaths. See for example Puerto Rico's note here https://covidtracking.com/data#state-pr

Michal,

Thank you for your quick and helpful reply!

That makes sense! Would it be fair to say that most states still report “confirmed” only (also when they do not comment on it)?

FYI: Other than for Puerto Rico, I saw that New York publishes separate figures (https://www1.nyc.gov/assets/doh/downloads/pdf/imm/covid-19-deaths-confirmed-probable-daily-04152020.pdf). As far as I am aware, these deaths cannot be fully allocated to specific days, so for my analysis I excluded these to not mess up the growth rates.

By the way, do you know if the Santa Clara Study uses “confirmed” deaths or “total” deaths? They do not specify it, but cite this website as a source (which does not specify it either, unfortunately): https://www.sccgov.org/sites/covid19/Pages/dashboard.aspx#cases

Thank you so much, Patrick

From: Michal M notifications@github.com Sent: 18 April 2020 14:39 To: COVID19Tracking/issues issues@noreply.github.com Cc: Patrick Mitschka patrick.mitschka@2binv.com; Author author@noreply.github.com Subject: Re: [COVID19Tracking/issues] Deaths: Confirmed vs. probable, new CDC guideline (#215)

Thanks, Patrick!

We are discussing how to handle the probable deaths as its own metric.

For now, we are including public notes about each state where the probable deaths are included and we do include the probable deaths within the overall deaths. See for example Puerto Rico's note here https://covidtracking.com/data#state-pr

[PR]<../../assets/github/images/1749380/79639222-587e5180-8158-11ea-8c6c-79358d44ba6f.PNG>

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/COVID19Tracking/issues/issues/215#issuecomment-615873438, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO4H6RNYXHFHLUJQGPWPR33RNGUONANCNFSM4MLIJP7A. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

Is there an update on this? The choice of different metrics to use is leading to a large discrepancy, for example Worldometers is reporting 18,929 deaths for New York State(similar numbers on JHU map, but harder to disentangle), while COVID Tracker is reporting 14,347 deaths here.

In particular, I'm interested in understanding whether the numbers of deaths reported will change by April 26 or May 8, since this is germane to a number of forecasts I'd like to do.

For reference, this is how worldometers handles it:

In accordance with new CDC guidelines:

New York: the numbers shown below include probable deaths (and, consequently, probable cases for the same number) as reported by New York City Maryland: includes probable deaths, as reported by the Department of Health (Section: "COVID-19 Statistics in Maryland") Wyoming: includes probable cases, as reported by the Department of Health Most state reports and dashboards (such as Texas, Pennsylvania, North Carolina, Missouri, etc.) lag behind or show incomplete data with respect to the data shown by individual counties on their official websites and dashboards, which is what we collecte and aggregate when possible to show the most updated and accurate number in the table below. We will soon publish state-level pages with graphs and the breakdown by county.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

Uploading a spreadsheet of all decreases (more than documented previously), current as of today: covidtracking_problemdates.xlsx

States affected: AK, AL, AZ, CO, DC, DE, FL, HI, IA, KS, MA, MD, MI, NJ, NM, NV, NY, OH, OK, PR, RI, SC, WI

This is my R code to calculate new cases and pull records that decreased from the prior day (any variable) AND the prior day's row for comparison:

library(tidyverse)
covidtracking %>% 
    arrange(state, date) %>% 
    group_by(state) %>% 
    mutate_at(vars(c("positive", "death", "total")), 
              list(new = ~ coalesce(. - lag(.), .))) %>%
    filter_at(vars(ends_with("new")), any_vars(. < 0 | lead(.) < 0)) %>% 
    ungroup()

I wonder if the problem is an error in the data source in which corrections for prior day results are included as adjustments on the day the error was discovered. This is common in the banking world because of the value of keeping past transactions immutable. It's poor practice for scientific data, however, because the test counts on a given day matter.

If the problem is deferred adjustment, the idea solution is to inform the data sources and ask for better quality data. Short of that, an corrective approach is to reverse the error to the best extent that the data allows: Where a daily result is negative, set that day to zero, and decrease the count of the previous day by the corresponding amount.

Such a correction would not be perfect since (a) you don't know for sure whether the error was from the previous day versus earlier and (b) it doesn't correct any of the cases where adjustments didn't cause negative result. Still, it leads to better data quality than making no correction and avoids the confusion of negative daily counts.

Hello, and thank you for helping us clean our data. Please see the following:

New York:

• [ ] 3/7 to 3/8 the total changed due to pending tests no longer being reported.
• [ ] 3/10 to 3/11 should be correct

Oklahoma

• [ ] 3/20 to 3/21 the variance in totals is tied to the unreliable Pending category. It may be that the data point was phased out and our data was affected by this transition;

Ohio

• [ ] 3/16 to 3/17 variation in data is due to Pending data no longer being published.

New Jersey

• [ ] 3/16 is infact incorrect:
• [ ] corrected

Hawaii

• [ ] Pending for 3/19 decrease per the state

Michigan stopped reporting pending data as of 3/17

Kansas stopped reporting pending data as of 3/11

Iowa stopped reporting pending data as of 3/14 Deleware stopped reporting pending data as of 3/17 DC has fluctuating pending data

Thank you for these clarifications, @careeningspace, and for working on providing these data so accessibly! In terms of cleaning up these records for use:

1. Has the NJ correction been applied to the live data?
2. Given the inconsistencies with pending data, do you foresee any problems with subtracting pending cases from the totals?

The NJ correction should be in the live feed. Going forward, our API will no longer be focusing on including Pending in our "Total". You can find more detail on our API Page

• [ ] totalTestResults - Calculated value (positive + negative) of total test results.
• [ ] total - DEPRECATED Will be removed in the future. (positive + negative + pending). Pending has been an unstable value and should not count in any totals.

As for subtracting historical "Pending" data - if you want a clean Total, you can sum Positive and Negative.

Great, thanks. That fixes problems in many states and leaves only AK, DC, HI, ID, KY, MI, NV, and SC with negative "increases": covidtracking_problemdates.xlsx

covidtracking %>% arrange(state, date) %>% group_by(state) %>% filter_at(vars(ends_with("Increase")), any_vars(. < 0 | lead(.) < 0)) %>% ungroup()

-[ ] Alaska has a period of data flux that needs more research 3/17 - 3/19: Data Log:

Daily Report:

State Data from 3/17 14:09 ET:

State Data from 3/17 18:00 ET:

State Data from 3/18:

Updated Daily after correction:

District of Columbia 3/10 - 3/11

• [ ] We do not have screen grabs from this time period
• [ ] It looks like DC changed how they were reporting data. I am going to make both days match

DC Before update:

DC after update:

• [ ] ID has an issue with a change in data from 3/18 to 3/19

• [ ] Screen cap of State Data from 3/18 14:04:

• [ ] Screencap of State Data from 3/19 14:04:

• [ ] The positives increase, while the total tests reported did not. Our methodology is to leave the negatives unchanged in this case. Fixed data below:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

hey @dimitrilucas thanks for the question!

it may be because our daily numbers are as of 4 pm ET. the total number on 3/16 for us was 4,019 while the cdc has 4,226, so we're definitely in the same ballpark. states report their data at different times on different days so i wouldn't focus too much on the precise number each day. a trend line would be more useful.

for more details on our methods, see https://covidtracking.com/newsroom-expert-faq.

Also I find the CDC's data rather opaque and confusing. There is no way to know exactly where it is coming from or what their update process is, nor an issue tracker to leave questions or suggestions...

Man there is just not an emoji to appropriately respond to this.

hey @dimitrilucas thanks for the question!

it may be because our daily numbers are as of 4 pm ET. the total number on 3/16 for us was 4,019 while the cdc has 4,226, so we're definitely in the same ballpark. states report their data at different times on different days so i wouldn't focus too much on the precise number each day. a trend line would be more useful.

for more details on our methods, see https://covidtracking.com/newsroom-expert-faq.

Also I find the CDC's data rather opaque and confusing. There is no way to know exactly where it is coming from or what their update process is, nor an issue tracker to leave questions or suggestions...

Man there is just not an emoji to appropriately respond to this.