Monitoring should answer whether data is usable, not just whether a run ended
The DP-700 monitoring domain includes ingestion, transformations, semantic model refresh, alerts, and diagnosis of pipelines, Dataflow Gen2, notebooks, Eventhouses, Eventstreams, T-SQL and OneLake shortcuts. These surfaces generate different evidence. A run marked successful means its configured activity exited appropriately; it does not by itself prove the intended output table is complete, has the correct schema or is current enough for consumers.
A retailer can run its midnight pipeline without an exception and still report yesterday’s total because the source connector returned an empty delta, a watermark advanced incorrectly or an upstream system published data late. The operator needs separate service-health and data-health signals. Define expected freshness, record counts, change volume, exception threshold and downstream refresh timing before deciding what a healthy run looks like.
Microsoft’s Fabric pipeline monitoring comparison distinguishes views for run history, investigations, alerts and capacity. Choose the supported experience available in the tenant; some newer monitoring interfaces remain in preview.
Start a pipeline failure from its failed activity, not its top-level label
A pipeline orchestrates activities such as Copy, Dataflow and Notebook execution. When it fails, locate the activity and its dependency context, then inspect the specific error and run identifiers. A parent-level ‘Failed’ state is useful for routing an incident but usually lacks enough information to determine whether the cause is source authentication, schema mismatch, transformation logic or destination capacity.
A connection timeout may be transient, and a bounded retry might be suitable. A source column unexpectedly changing from integer to structured JSON needs a schema decision, not repeated retries. A destination conflict may indicate overlapping runs or a previous operation that left incompatible state. Adjusting all retry counts upward can worsen contention and obscure the actual defect.
A reliable Fabric pipeline needs explicit retry behavior, idempotent transformations and durable progress markers. When a run fails, use orchestration logs to locate the failed boundary, check whether earlier steps committed and choose a retry that cannot duplicate or silently drop data. Monitoring does not create reliability by itself; it supplies the evidence required for a safe recovery decision.
Dataflow Gen2 errors often originate in a specific Power Query or destination step
A Dataflow Gen2 item can fail while connecting to a source, evaluating a transformation step, resolving credentials or writing a destination. The useful investigation follows those steps and the documented refresh details. A source credential expiring is not a reason to weaken row security at the destination. A type-conversion failure on one malformed record may require quarantine and a defined policy for invalid values.
Check whether an unsupported or nonfolding transformation causes unexpected data movement or query duration. A dataflow that refreshes but writes zero rows may indicate a faulty filter or source view. Distinguish a refresh status from an output test that compares row counts and business keys to expected control totals.
If multiple dataflows share a source, examine whether the failure began after a common change to permissions, schema or endpoint. Coordinated diagnosis is more effective than having each article or data engineer independently override security or replace null values with fabricated defaults.
A practical Dataflow Gen2 triage begins with the earliest failing step and a known sample record. If a Power Query conversion rejects ’31/02/2026′ as a date, the source is invalid and changing a credential cannot help. If a source query times out after a recently added join, evaluate folding and predicate placement rather than automatically raising all timeouts. If dataflow execution succeeds yet its lakehouse destination is empty, verify whether destination write configuration, schema and source filtering produced zero rows. Each symptom needs evidence at the right interface, not an indiscriminate retry.
Notebook failures need job, environment and data evidence
A Fabric notebook can fail before the transformation runs because of environment or library resolution, during execution because of code or Spark resource problems, or on write because the destination is unavailable. Inspect the execution context and error stage. A successful interactive test with a developer identity does not prove a scheduled run using a different identity and environment will behave identically.
For Spark code, investigate stage duration, skew, shuffle volume and executor failures when performance or resource errors occur. For data correctness, check the actual transformed dataframe or destination table. A job that completes after filtering out all records is not an operational success if the contract expects 100,000 orders. Build a validation condition that can fail the workflow for an unacceptable output.
The workspace and Spark settings topic explains runtime configuration, while this monitoring objective asks how an operator uses observed execution evidence to isolate a problem without a risky broad change.
Eventstream and Eventhouse failures affect different parts of the path
An Eventstream source may stop sending events, transformations can discard or misroute events, and a destination can experience ingestion or schema issues. An Eventhouse query can separately fail because of permissions, syntax, resources or data layout. One ‘no data’ symptom could arise at any of these stages. Follow a known event through the source, stream, stored records and query result before assuming a query optimizer is at fault.
Timestamps are especially important. If a device publishes event time in seconds but a transformation interprets it as milliseconds, a dashboard may show apparently no current events even though records are stored. Compare origin time, ingestion time, query filters and window boundaries. Retained evidence from the original source helps establish whether the problem began before or after ingestion.
Fabric Real-Time Intelligence components have supported monitoring capabilities and performance or capacity limits that can evolve. Report missing visibility as a limitation, not a green health score. The streaming window article describes the event-time decisions that monitoring must verify.
SQL and OneLake shortcut errors need separate boundary tests
T-SQL errors can reflect syntax, missing objects, permission denial, incompatible schema or runtime execution limits. A successful connection only proves the initial path is open; it does not show that a principal can select or update the intended resource under the engine’s supported semantics. Capture the specific query, object, identity and error rather than broadening production permissions to see whether the failure disappears.
A OneLake shortcut may appear in the lakehouse explorer while its backing path is unavailable or insufficiently authorized. Diagnose shortcut definition, target location, source credential and expected format separately from local workspace visibility. A broken shortcut is not repaired by adding a new unrelated copy unless the architecture legitimately changes to a different storage pattern.
Where a notebook consumes a shortcut and writes a warehouse, trace the failure through each identity and store. A source failure could manifest as a downstream empty table, so the absence of a warehouse error does not mean the source ingestion was sound.
OneLake shortcut troubleshooting also requires checking producer change windows. An upstream source owner can legitimately reorganize a data path during maintenance, leaving the consumer’s shortcut stale while the underlying data remains available under a new location. Rather than converting every failing shortcut into an independent copied dataset, agree on a path-change and schema-change contract with the producer, document the exact failure, and update the reference in a controlled release. Then verify consumer query output and access boundaries with a known test principal. This keeps storage reuse manageable instead of creating hidden replication.
Alerting must be specific enough to support a safe response
An alert should have a condition, scope, responsible operator, expected action and evidence link. A generic ‘pipeline failed once’ notification might be noisy if transient retries are part of normal operation; no alert on an empty daily output might be dangerous. Balance technical failure signals with data freshness, quality and downstream business importance. Incident severity should reflect actual consumer impact, not only the existence of one red status icon.
For a finance pipeline, alert when the agreed reporting boundary has passed without a validated gold table, when a repeated source failure exhausts the safe retry policy, or when a control total deviates beyond documented tolerance. Use separate signals for a failed job and a suspicious data result. Keep alert history and operator decisions so recurring defects can be traced to a source rather than hidden by manual reruns.
Automation may open a ticket or notify an owner without requiring production write privileges. More consequential actions—such as deleting a table or revoking a broad connection—need stronger authorization and suitable rollback. The monitoring system should not silently assume every detected symptom has one safe automatic fix.
Build a diagnostic runbook tied to the failing stage
A strong runbook starts by identifying the workload and last known good result, then collecting source status, run and activity IDs, error details, relevant input/output counts and time boundaries. It distinguishes a retryable service interruption from a data-contract defect and explicitly states when a replay can be performed. Operators should know which checkpoints might be affected and which downstream consumers need notice.
After remediation, validation should include actual data output and consumer-facing freshness, not only a new green run. For a corrected late order, confirm the fact table and derived aggregate contain the intended change without duplicates. For restored event ingestion, confirm a known safe event reaches the query result in the correct window.
DP-700 readiness means using evidence to isolate the failure and choosing a safe remedy at the correct boundary. A hypothetical runbook can demonstrate that reasoning, but do not present it as real production incident evidence when no monitoring system was actually accessed.