Michigan

Changes (10).txt

Changes (10).txt

Changes (9).txt

Changes (9).txt

BEFORE:

AFTER:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

Done in #701, closing issue.

BEFORE:

AFTER:

Previous Backfill GH issue https://github.com/COVID19Tracking/issues/issues/487

Before and After values are in the attached sheet. Probable-Confirmed-Total Cases Timeseries Backfill MI.xlsx

Upon reviewing screenshots, numbers from the patient census were used: https://covid-tracking-project-data.s3.us-east-1.amazonaws.com/state_screenshots/MI/MI-secondary-20200621-183853.png After: Before:

A second look was taken at 6/15-/30. Fixed using screenshots of the patient census table. Other issues found: Inaccurate vents on 6/21 (298 -> 209) Before: After:

We updated the data based on XL provided by Michigan's DHHS See before and after snapshots here https://docs.google.com/spreadsheets/d/1n0dOR8Dxjtg_usQdgbYe4LuDPURi_HK6WFL3wvawRn0/edit#gid=500776026

Hi @muamichali, thanks for opening the issue. After reviewing our screenshots of Michigan's COVID-19 site, I'm confirming Michigan reported the # of Inpatients as 953 on 5/24, 5/25 and 5/26.

In addition, on 5/26, Michigan also updated the # of Inpatients from 953 to 882 after we published. We publish the daily stats every afternoon. If a state updates the number of tests or cases after that time, we catch that update on the following day.

Thank you, Camille

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

Fixing it now: Before After:

Updated, Quang!

Before

After

Thank you! This looks good.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

Hi @saewoonam, Thanks for bringing this to our attention. We've since backfilled the MI data for 3/28 so this issue is no longer relevant. However, this lag was likely due to the data for MI updating after our publication cutoff. We publish the daily stats every afternoon. If a state updates the number of tests or cases after that time, we catch that update on the following day.

Potential duplicate of #79

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

Please reference issue #173 for a more comprehensive response.

Potential duplicate of #80

Hello,

We did update Michigan data after the state updated its reporting process. Please find more detailed information at #173

Hi, @rmontroy, I'm reassigning this to the general issues queue so that the data team sees it.

Michigan is not being updated according to the covid19tracking web page because it is reporting specimens rather than people tested. I urge you to report the specimen counts (with caveats) for the following reasons:

1. Other states are very likely double-counting people who have been tested multiple times.
2. Michigan’s confirmed case count is close to the reported positive count, suggesting they are not systematically taking multiple specimens (e.g. nose and mouth) per person.
3. Ramp up in total test capacity is important to track in its own right.

This seems to have been resolved. See the Michigan page on the site.

• The data grade is now a B, accounting for reporting negatives.
• The data is noted to be reporting "total specimens"

Uploading a spreadsheet of all decreases (more than documented previously), current as of today: covidtracking_problemdates.xlsx

States affected: AK, AL, AZ, CO, DC, DE, FL, HI, IA, KS, MA, MD, MI, NJ, NM, NV, NY, OH, OK, PR, RI, SC, WI

This is my R code to calculate new cases and pull records that decreased from the prior day (any variable) AND the prior day's row for comparison:

library(tidyverse)
covidtracking %>% 
    arrange(state, date) %>% 
    group_by(state) %>% 
    mutate_at(vars(c("positive", "death", "total")), 
              list(new = ~ coalesce(. - lag(.), .))) %>%
    filter_at(vars(ends_with("new")), any_vars(. < 0 | lead(.) < 0)) %>% 
    ungroup()

I wonder if the problem is an error in the data source in which corrections for prior day results are included as adjustments on the day the error was discovered. This is common in the banking world because of the value of keeping past transactions immutable. It's poor practice for scientific data, however, because the test counts on a given day matter.

If the problem is deferred adjustment, the idea solution is to inform the data sources and ask for better quality data. Short of that, an corrective approach is to reverse the error to the best extent that the data allows: Where a daily result is negative, set that day to zero, and decrease the count of the previous day by the corresponding amount.

Such a correction would not be perfect since (a) you don't know for sure whether the error was from the previous day versus earlier and (b) it doesn't correct any of the cases where adjustments didn't cause negative result. Still, it leads to better data quality than making no correction and avoids the confusion of negative daily counts.

Hello, and thank you for helping us clean our data. Please see the following:

New York:

• [ ] 3/7 to 3/8 the total changed due to pending tests no longer being reported.
• [ ] 3/10 to 3/11 should be correct

Oklahoma

• [ ] 3/20 to 3/21 the variance in totals is tied to the unreliable Pending category. It may be that the data point was phased out and our data was affected by this transition;

Ohio

• [ ] 3/16 to 3/17 variation in data is due to Pending data no longer being published.

New Jersey

• [ ] 3/16 is infact incorrect:
• [ ] corrected

Hawaii

• [ ] Pending for 3/19 decrease per the state

Michigan stopped reporting pending data as of 3/17

Kansas stopped reporting pending data as of 3/11

Iowa stopped reporting pending data as of 3/14 Deleware stopped reporting pending data as of 3/17 DC has fluctuating pending data

Thank you for these clarifications, @careeningspace, and for working on providing these data so accessibly! In terms of cleaning up these records for use:

1. Has the NJ correction been applied to the live data?
2. Given the inconsistencies with pending data, do you foresee any problems with subtracting pending cases from the totals?

The NJ correction should be in the live feed. Going forward, our API will no longer be focusing on including Pending in our "Total". You can find more detail on our API Page

• [ ] totalTestResults - Calculated value (positive + negative) of total test results.
• [ ] total - DEPRECATED Will be removed in the future. (positive + negative + pending). Pending has been an unstable value and should not count in any totals.

As for subtracting historical "Pending" data - if you want a clean Total, you can sum Positive and Negative.

Great, thanks. That fixes problems in many states and leaves only AK, DC, HI, ID, KY, MI, NV, and SC with negative "increases": covidtracking_problemdates.xlsx

covidtracking %>% arrange(state, date) %>% group_by(state) %>% filter_at(vars(ends_with("Increase")), any_vars(. < 0 | lead(.) < 0)) %>% ungroup()

-[ ] Alaska has a period of data flux that needs more research 3/17 - 3/19: Data Log:

Daily Report:

State Data from 3/17 14:09 ET:

State Data from 3/17 18:00 ET:

State Data from 3/18:

Updated Daily after correction:

District of Columbia 3/10 - 3/11

• [ ] We do not have screen grabs from this time period
• [ ] It looks like DC changed how they were reporting data. I am going to make both days match

DC Before update:

DC after update:

• [ ] ID has an issue with a change in data from 3/18 to 3/19

• [ ] Screen cap of State Data from 3/18 14:04:

• [ ] Screencap of State Data from 3/19 14:04:

• [ ] The positives increase, while the total tests reported did not. Our methodology is to leave the negatives unchanged in this case. Fixed data below:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

This issue has been closed because it was stale for 15 days, and there was no further activity on it for 10 days. You can feel free to re-open it if the issue is important, and label it as "not stale."

I'm trying to understand the issue.

Julia archives specific states. Here is MI: http://covid-data-archive.s3-website.us-east-2.amazonaws.com/state_screenshots/MI.

Did you want an additional page capture for MI (which we can do)? Or an alternative time for everything (which we can do)? Or are you looking at a completely different set of pages (so it is not a task for us)?

I'm trying to understand the issue.

Julia archives specific states. Here is MI: http://covid-data-archive.s3-website.us-east-2.amazonaws.com/state_screenshots/MI.

Did you want an additional page capture for MI (which we can do)? Or an alternative time for everything (which we can do)? Or are you looking at a completely different set of pages (so it is not a task for us)?

I am asking screenshots for

https://www.michigan.gov/coronavirus (not saved)

to be taken in addition to

https://www.michigan.gov/coronavirus/0,9753,7-406-98163-520743--,00.html (saved)

the mentioning of time is if it would make you guys' lives easier, the 5am capture isn't needed bc MI only updates once / day.

and also to update the footnote regarding the two sites being out of date, they aren't. I believe that's a misunderstanding. I explained why the two pages differ in #29 .

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

We are currently capturing two URLs for MI

https://www.michigan.gov/coronavirus/0,9753,7-406-98163_98173---,00.html and https://www.michigan.gov/coronavirus/0,9753,7-406-98159-523641--,00.html

You can see them at http://covid-tracking-project-data.s3-website.us-east-1.amazonaws.com/state_screenshots/MI/

I hope this is sufficient to close this issue