* Set SKIN = nat

IT notes related to the N2QOD project

Process Review

14 July 2017


Steps to produce the N2QOD reports

Log on to my account on biostat059.dhcp.mc.vanderbilt.edu

Set default to the ~/projects directory. The ~/projects directory contains all the code (R, Python, etc.) and the downloaded data.
$ cd ~/project

Download the data from the REDCap tables to the local computer. This step downloads ALL the data so it can take a while to complete.
$ ./download_data

If this step indicates any errors then we can't continue before fixing the problem.
$ ./check_changes

Now, actually produce the reports.

For the 1st or 3rd Friday we make the Numbers (Enrollment)
reports
For the 2nd or 4th Friday we make the Audit reports
 $ make enroll_reports 
 $ make audit_reports 
Zip up the reports.
This produces a file reports-audityyyy-mm-dd.zip
 $ ./zip_numbers_reports yyyy-mm-dd 

where yyyy-mm-dd is the date that the data was downloaded.
Zip up the reports.
This produces a file reports-audityyyy-mm-dd.zip
 $ ./zip_audit_reports yyyy-mm-dd 

where yyyy-mm-dd is the date that the data was downloaded.

Send the reports (reports-audityyyy-mm-dd.zip) to Swygert, Kristin Archer <kristin.archer@vanderbilt.edu>, Clemons, Lori <lori.clemons@Vanderbilt.Edu>, and claudia.davidson@vanderbilt.edu via Vanderbilt Accellion. This needs to be done by 11:00 AM on the Thursday before the Friday due date.

On Friday, upload the zipped reports to the database. Do not do this step until Kristin and Lori have had a chance to review the reports.

For the 1st or 3rd Friday we make the Numbers (Enrollment)
reports
For the 2nd or 4th Friday we make the Audit reports
 $ ./upload_numbers nn month yy yyyy-mm-dd
 $ ./upload_audit nn month yy yyyy-mm-dd

Where:
    • nn is either "12" or "34".
    • use "12" for 1st Friday numbers reports
    • use "12" for 2nd Friday audit reports
    • use "34" for 3rd Friday numbers reports
    • use "34" for 4th Friday audit reports
  • month is the name of the month ("january", "february",...,"december"). Not abbreviated and all lower case.
  • yy is the year in 2 digits (e.g. "17" for 2017)
  • yyyy-mm-dd is the date that the data was pulled.

22 Feb 2016

Regular daily meeting today. We talked about incorrectly formatted dates being entered by staff in the field. Item #1 has been added to Thomas's to do list.

To do for Thomas:
  1. Add code to detect incorrectly formatted dates. Evidently, the date format checking in Redcap does not always work which allows personnel in the field to enter incorrectly formatted dates. Currently, these end up as blank fields after the data conversion step of the reporting process is done. Thomas will add code to detect these bad dates and note them in the reports.
  2. Refresh Dale's mirror of the code
  3. Produce a diagram for the process used to produce the PQRS surgeon report
  4. Produce a diagram for the process used to produce the PQRS XML file
  5. Produce a script that Dale can follow to produce the PQRS surgeon report
  6. Produce a script that Dale can follow to produce the PQRS XML file

21 Oct 2016

Thomas and I met today and he described the process for producing some of the PQRS/QCDR reports. Specifically, we discussed the "surgeon report" and the production of the xml file. The PQRS/QCDR reports and files are yearly in scope. After the end of the year (31 Dec), final versions of these reports/files will be produced for submission to the N2QOD authorities. In the meantime, they are being used as "year to date" reports.

There are basically 4 reports in the PQRS/QCDR area:
  1. surgeon report
  2. QCDR patient reports
  3. QCR final report
  4. xml file production

Reports 1 and 4 are currently (21-Oct-2016) active. Code for reports 2 and 3 are written but not thoroughly tested.

"PQRS" means Physician Quality Reporting System "QCDR" means Qualified Clinical Data Registry

To do for Thomas:
  • Add code to detect incorrectly formatted dates. Evidently, the date format checking in Redcap does not always work allowing personnel in the field to enter incorrectly formatted dates. Currently, these end up as blank fields after the data conversion step of the reporting process is done. Thomas will add code to detect these bad dates and note them in the reports.
  • Refresh Dale's mirror of the code
  • Produce a diagram for the process used to produce the PQRS surgeon report
  • Produce a diagram for the process used to produce the PQRS XML file
  • Produce a script that Dale can follow to produce the PQRS surgeon report
  • Produce a script that Dale can follow to produce the PQRS XML file

1 Sep 2016

Reviewed points of emphasis today.

Points of emphasis:
  • Make a test run of numbers and audit reports (Dale) (added 9/12/2016). The goal here is to make sure Dale still knows how to do this.
  • Make a report that lists the participating centers as of the time the report is run (added 9/12/2016). Came up when we were thinking about the issue where a center did not receive a PQRS report and it is not clear why.
  • Document (script) production of XML submission files. The goal is to make it possible for a backup person to produce these.
  • Produce flow chart of process of production of XML submission files
  • PQRS scripting and documentation
  • Documentation and scripting of preparation of data for quarterly reports. The goal is to make it possible for a backup person to produce these.
  • More thoroughly comment all the code
  • Pay special attention to commenting code that uses regular expressions
  • Consolidate common code
  • Send an updated flow diagram
  • Continue to work on scripts/procedures so backup can run reports when Thomas is on vacation.
  • Use a special comment (e.g. ##HARDCODED) to call attention to code that contains hardcoded values. These are possible sources of failure if the condition that requires the hard coding changes. For example, there is a place where a surgeon can be associated with up to 9 hospitals. A periodic sweep of all hard coded values should be made.
  • Remove as many uses of hard coded values as possible.

11 Aug 2016

  • Thomas will copy latest code to my directory.
  • I will see if I can run numbers reports
  • Thomas will document procedure to produce data for quarterly reports.

21 July 2016

  • Need to know about quarterly reports. What are they? What is the schedule? How produced? Etc.
  • It appears that a schema change was made in REDCap between 7/19/2016 and today (7/21/2016). This is causing the report build to fail because there are different variables in one or more of the tables. I was surprised that there was a schema change when the programmer is out of town and a backup is doing the reports?

11 July 2016


Steps to produce the N2QOD reports

Log on to my account on biostat059.dhcp.mc.vanderbilt.edu

Set default to the ~/projects directory. The ~/projects directory contains all the code (R, Python, etc.) and the downloaded data.
$ cd ~/project

Download the data from the REDCap tables to the local computer. This step downloads ALL the data so it can take a while to complete.
$ ./download_data

If this step indicates any errors then we can't continue before fixing the problem.
$ ./check_changes

Now, actually produce the reports.

For the 2nd or 4th Friday we make the Audit reports
$ make audit_reports

Zip up the reports. This produces a file reports-audityyyy-mm-dd.zip
$ ./zip_audit_reports yyyy-mm-dd

where yyyy-mm-dd is the date that the data was downloaded.

Send the reports (reports-audityyyy-mm-dd.zip) to Swygert, Kristin Archer <kristin.archer@vanderbilt.edu>, Clemons, Lori <lori.clemons@Vanderbilt.Edu>, and claudia.davidson@vanderbilt.edu via Vanderbilt Accellion. This needs to be done by 11:00 AM on the Thursday before the Friday due date.

On Friday, upload the zipped reports to the database. Do not do this step until Kristin and Lori have had a chance to review the reports.
$ ./upload_audit nn month yy yyyy-mm-dd
  • nn is either "12" or "34".
    • use "12" for 1st Friday numbers reports
    • use "12" for 2nd Friday audit reports
    • use "34" for 3rd Friday numbers reports
    • use "34" for 4th Friday audit reports
  • month is the name of the month ("january", "february",...,"december"). Not abbreviated and all lower case.
  • yy is the year in 2 digits (e.g. "16" for 2016)
  • yyyy-mm-dd is the date that the data was pulled.

Produce the out of range report. This is a manual step. Repeat for lumbar, cervical, and cerebrovascular.
$ R --slave -f out_of_range_report_lumbar.R
$ R --slave -f out_of_range_report_cervical.R
$ R --slave -f out_of_range_report_cerebrovascular.R

Copy and paste the output that is produced into a LibreOffice calc spreadsheet. Make one tab for each report can call the tabs "lumbar", "cervical", and "cerebrovascular". Call the calc file "out_of_range_report.xls" (Microsoft Excel format). When pasting into the spreadsheet, use the Edit tab and select "Paste special". Make sure that in the Text Import dialog "Space" is selected as a separator and "Merge delimiters" is selected. Like this:


Screenshot_from_2016-07-13_10:24:02.png

Send this as an email attachment to Clemons, Lori <lori.clemons@Vanderbilt.Edu>.

Notes on report production

What is producing the files with the extension ".csve"?
$ ~/project$ find . -type f -name "*csve"
./patient_lumbar.csve
./n2qod_pqrs.csve
./patient_cervical.csve
./surgeon.csve
Answer: There is a sed script in download_data.
$ ~/project$ cat download_data
#!/bin/bash

make update

sed -ie 's/^AtlanticNS/Atlantic_NS/' patient_lumbar.csv patient_cervical.csv

make archive
Notice the -ie qualifier. This causes sed to edit files in place and makes a backup using the supplied suffix in the file name. the suffix is "e" in this case. Hence the file names with "csve" as the extensions.

Thomas will produce the reports for July 15th.

For Friday July 22nd I will be producing the reports. That is a 4th Friday so it will be the Quality Audit reports that are generated. In the steps below we are showing only the Quality Audit reports productions. This should be expanded to show the Numbers reports.

Reports are produced on Fridays according to this schedule:

1st and 3rd Friday produce the numbers reports
2nd and 4th Friday produce the audit reports

There are no reports on any 5th Fridays.

It is standard practice to pull the data on Thursday before the reports are due. The reports should be completed before 11:00 AM on Thursday and sent to Kristin and Lori for their review.

The ~/project directory contains all the code (R, Python, etc.) and the downloaded data.

The ~/project/reports directory contains the reports that are produced by these steps. This directory can be cleaned periodically.

8 June 2016

Discussed "rules" files. These are the R programs that contain the checks that are performed, the results of which are displayed in the reports. These rules files are each a very long sequence of checks.

Points of emphasis:
  • More thoroughly comment all the code
  • Pay special attention to commenting code that uses regular expressions
  • Consolidate common code
  • Send an updated flow diagram
  • Continue to work on scripts/procedures so backup can run reports when Thomas is on vacation.
  • Use a special comment (e.g. ##HARDCODED) to call attention to code that contains hardcoded values. These are possible sources of failure if the condition that requires the hard coding changes. For example, there is a place where a surgeon can be associated with up to 9 hospitals. A periodic sweep of all hard coded values should be made.
  • Remove as many uses of hard coded values as possible.

18 May 2016 and 16 May 2016

I was out of the office on Monday and Thomas had a deadline on Wednesday so we didn't do any code review on these days. We will resume next week.

11 May 2016

Discussed the lumbar deformity reports. The programs "create_patient_lumbar_reports.R" and "create_patient_lumbar_deformity_reports.R" have much common code. Thomas will combine these programs into one. The combined program will take an argument that indicates whether it is to produce the lumbar reports or the lumbar deformity reports. Approximately 200 lines of code will be eliminated. It is good to eliminate code!

9 May 2016

Discussed the lumbar numbers reports.

4 May 2016

Thomas has produced some diagrams that explain the flow of the process used to produce the reports. This is very helpful to get a high level view of the process.

I am also trying to get a better handle on the naming being used. There are 4 "modules" and several different kinds of reports
  1. lumbar module
    1. weekly numbers report "numbers report" (PDF) 1&3 Fridays
    2. NPA weekly numbers report "NPA numbers report" (PDF) 1&3 Fridays
    3. NPA accrual numbers report "accrual numbers spreadsheet" (a CSV file) 1&3 Fridays
    4. Quality Audit reports (2 & 4 Fridays) "audit report"
  2. (lumbar) deformity module
    1. weekly numbers report "numbers report" (PDF) 1&3 Fridays
    2. NPA weekly numbers report "NPA numbers report" (PDF) 1&3 Fridays
    3. NPA accrual numbers report "accrual numbers spreadsheet" (a CSV file) 1&3 Fridays
  3. cervical module
    1. weekly numbers report "numbers report" (PDF) 1&3 Fridays
    2. NPA weekly numbers report "NPA numbers report" (PDF) 1&3 Fridays
    3. NPA accrual numbers report "accrual numbers spreadsheet" (a CSV file) 1&3 Fridays
    4. Quality Audit reports (2 & 4 Fridays) "audit report"
  4. cerebrovascular module
    1. weekly numbers report "numbers report" (PDF) 1&3 Fridays
    2. NPA weekly numbers report "NPA numbers report" (PDF) 1&3 Fridays
    3. NPA accrual numbers report "accrual numbers spreadsheet" (a CSV file) 1&3 Fridays
    4. Quality Audit reports (2 & 4 Fridays) "audit report"

2 May 2016

I asked that the code be more thoroughly documented.

We discussed making a script that could be used to produce the reports rather than typing a number of commands manually. The goal is to avoid errors. A secondary goal is to make it possible for another person to run the reports if Thomas is absent.
cd work_shop/N2QOD/project
./download_data
./check_changes

make audit_reports
make enroll_reports
Here there is a manual step to zip up the reports and send for review. This should be automated.
./upload_audit ## month yr yyyy-mm-dd
./upload_numbers ## month yr yyyy-mm-dd
where
  • "##" is either "12" or "34". For numbers reports "12"=1st Friday reports, "34"=3rd Friday reports. For audit reports "12"=2nd Friday reports, "34"=4th Friday reports.
  • "month" is the month (march, april, may, etc.)
  • "yr" is the 2-digit year (e.g. 16)
  • "yyyy-mm-dd" is the date used to select the files containing the reports to be uploaded

Issues and Resolutions

11 May 2016

We discussed the recent issue in which an out of date report got uploaded. Thomas was doing some debugging and had a date temporarily hard coded in an bash script. When he reverted to the programmatic dates he overlooked one line and the fixed date remained. This is a classic case of using production code when making changes and should be avoided. We are going to change the version management organization so that there is a "production" branch and a "development" branch. Changes will be made and tested in the development branch. Only code tested and intended for production will be merged into the production branch.

-- DalePlummer - 14 Jul 2017
Topic revision: r1 - 14 Jul 2017, DalePlummer
 

This site is powered by FoswikiCopyright © 2013-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback