Running Demo Merge Project

1. Generate the test code

cd  test_data_generator
python generate_merge_test_data.py 100
cp *.csv ../../importdata/
cd ..

2. Ensure the following tables have been created in the merge_utilities schema. If not then run dag 01_setup_provider_schema to create them
   • match_confidence_sets
   • ui_set_membership
   • ui_set_status
3. Get the test data into landing by running these dags:
   • 02_import_and_conform_source1_to_v12
   • 03_import_data_into_landing (with source Affordablecare)
4. Add this row to monarch_config.remap_table

parent_object_name	object_name	remap_object_name	id_col	enabled
account	account	account	parentid	true

4. Generate match confidence sets by running dag merge_preprocess. At this point you can navigate to the merge UI and see some data (everything except merge preview)
5. Generate merge instructions and remap instructions by running dag provider_merge
6. Generate the merge preview by running dag 04_data_quality_and_mastering
7. Go to the merge UI and see the full view (including merge preview)

Troubleshooting

• If the merge_preprocess schema isn't showing up, double check the that the ports are set correctly in src/custom/dbt/profiles.yml