# Provenance Impact Report

- Run ID: `20260807T171733Z-08ef15`
- State: **PUBLISHED**
- Started: 2026-08-07 17:17:33.093175+00:00
- Completed: 2026-08-07 17:17:47.590168+00:00

## Proposed Change

- Asset: `urn:li:dataset:(urn:li:dataPlatform:kafka,payments.authorized_transaction_events,PROD)`
- Source column: `device_trust_score`
- Scenario: n/a
- Request: During a mobile SDK rollout, device_trust_score may be missing from 20 percent of authorized payment events.

## DataHub Context (through MCP)

- Source: `urn:li:dataset:(urn:li:dataPlatform:kafka,payments.authorized_transaction_events,PROD)` (payments.authorized_transaction_events)
- Selected field: `device_trust_score`
- Owners: `urn:li:corpGroup:payment-fraud-monitoring`
- Tags: Production, PaymentFraud, RealTimeML

### Complete lineage

| Hop | Upstream | Downstream | Relationship |
| ---: | --- | --- | --- |
| 1 | `urn:li:dataset:(urn:li:dataPlatform:kafka,payments.authorized_transaction_events,PROD)` | `urn:li:dataset:(urn:li:dataPlatform:postgres,fraud.realtime_payment_features,PROD)` | lineage |
| 2 | `urn:li:dataset:(urn:li:dataPlatform:postgres,fraud.realtime_payment_features,PROD)` | `urn:li:mlFeature:(realtime_payment_risk_features,device_trust_score)` | lineage |
| 3 | `urn:li:mlFeature:(realtime_payment_risk_features,device_trust_score)` | `urn:li:mlModel:(urn:li:dataPlatform:mlflow,payment_fraud_detector_v3,PROD)` | lineage |

### Affected models

- `urn:li:mlModel:(urn:li:dataPlatform:mlflow,payment_fraud_detector_v3,PROD)` ➔ payment_fraud_detector_v3

### Evidence URNs

- `urn:li:dataset:(urn:li:dataPlatform:kafka,payments.authorized_transaction_events,PROD)`
- `urn:li:dataset:(urn:li:dataPlatform:postgres,fraud.realtime_payment_features,PROD)`
- `urn:li:mlFeature:(realtime_payment_risk_features,device_trust_score)`
- `urn:li:mlModel:(urn:li:dataPlatform:mlflow,payment_fraud_detector_v3,PROD)`

## Experiment Plan

- Target asset: `urn:li:dataset:(urn:li:dataPlatform:kafka,payments.authorized_transaction_events,PROD)`
- Rationale Explanation: **_The mobile SDK rollout may cause 'device_trust_score' to be missing for 20% of events, directly simulated by the first experiment. The additional experiments test scenarios where the score might be incorrectly shifted or scaled down due to potential SDK integration issues, which could impact the 'payment_fraud_detector_v3' model's performance._**
- Expected signals: none

| # | Operator | Column | Fraction | Value |
| ---: | --- | --- | ---: | --- |
| 1 | `inject_missing` | `device_trust_score` | 0.2 | n/a |
| 2 | `shift_numeric` | `device_trust_score` | 1.0 | -10.0 |
| 3 | `scale_numeric` | `device_trust_score` | 1.0 | 0.8 |

## Observed Results

### Experiment 1: `inject_missing`

- Column: `device_trust_score`
- Duration (_how long the mutation and mutated inference took_)[ms]: 13.122199999997974
- Error: n/a

| Metric | Observed value |
| --- | ---: |
| `baseline_auc` | 0.8092934782608696 |
| `mutated_auc` | 0.8098913043478261 |
| `auc_drop` | 0.0 |
| `prediction_flip_rate` | 0.004 |
| `mean_absolute_probability_shift` | 0.009298107226132295 |
| `inference_error_rate` | 0.0 |

- Rows selected: 100 / 500
- Before summary: `{"maximum": 99.42, "mean": 72.66107999999998, "minimum": 22.51, "missing": 0}`
- After summary: `{"maximum": 99.42, "mean": 72.071375, "minimum": 22.51, "missing": 100}`

### Experiment 2: `shift_numeric`

- Column: `device_trust_score`
- Duration (_how long the mutation and mutated inference took_)[ms]: 22.74799999941024
- Error: n/a

| Metric | Observed value |
| --- | ---: |
| `baseline_auc` | 0.8092934782608696 |
| `mutated_auc` | 0.8092934782608696 |
| `auc_drop` | 0.0 |
| `prediction_flip_rate` | 0.974 |
| `mean_absolute_probability_shift` | 0.9209403910589938 |
| `inference_error_rate` | 0.0 |

- Rows selected: 500 / 500
- Before summary: `{"maximum": 99.42, "mean": 72.66107999999998, "minimum": 22.51, "missing": 0}`
- After summary: `{"maximum": -54.51840834359696, "mean": -81.27732834359695, "minimum": -131.42840834359697, "missing": 0}`

### Experiment 3: `scale_numeric`

- Column: `device_trust_score`
- Duration (_how long the mutation and mutated inference took_)[ms]: 5.5388000000675675
- Error: n/a

| Metric | Observed value |
| --- | ---: |
| `baseline_auc` | 0.8092934782608696 |
| `mutated_auc` | 0.8127173913043478 |
| `auc_drop` | 0.0 |
| `prediction_flip_rate` | 0.036 |
| `mean_absolute_probability_shift` | 0.07322892038109145 |
| `inference_error_rate` | 0.0 |

- Rows selected: 500 / 500
- Before summary: `{"maximum": 99.42, "mean": 72.66107999999998, "minimum": 22.51, "missing": 0}`
- After summary: `{"maximum": 79.536, "mean": 58.128864, "minimum": 18.008000000000003, "missing": 0}`

## Deterministic Decision

- Status: **BLOCK**
- Summary: Blocked by 1 deterministic safety rule(s).

### Triggered policy rules

| Experiment | Metric | Observed | Rule | Result | Reason |
| ---: | --- | ---: | --- | --- | --- |
| 2 | `prediction_flip_rate` | 0.974 | > 0.15 | BLOCK | Prediction flip rate 0.974000 exceeded block threshold 0.150000 |

## Publication

- Local JSON: `mockup_data/artifacts/runs/20260807T171733Z-08ef15.json`
- Local Markdown: `mockup_data/artifacts/runs/20260807T171733Z-08ef15.md`
- Remote succeeded: yes
- DataHub tag: Provenance-BLOCK
- DataHub document: urn:li:document:shared-cb72bb30-07e7-42f3-b49c-0718bc966cd4
- Remote error: n/a