The DP-700 performance domain covers lakehouse tables, pipelines, warehouses, Eventstreams, Eventhouses, Spark and query tuning. No single configuration accelerates every workload. A slow analytical job can be limited by storage layout, source rate, compute scheduling, excessive shuffles, data skew, query predicates or simply an unnecessary transformation. Begin with a baseline: elapsed time, records processed, bytes read or written, capacity pressure and whether the result is accurate.<\/p>\n
Consider a daily orders table whose query time doubled after an ingestion redesign. The cause might be thousands of small Delta files, a changed join cardinality, a missing selective predicate or competing workloads. Increasing Fabric capacity before checking those indicators can raise cost without resolving the architecture defect. Conversely, spending days rewriting SQL when the capacity is consistently throttled may be equally unproductive.<\/p>\n
Microsoft’s cross-workload table optimization guidance<\/a> explains why physical layout ownership differs among lakehouse, warehouse and mirrored data. This boundary matters: not every item exposes the same maintenance controls.<\/p>\n Delta Lake stores transactional table data in files tracked by a log. Frequent small writes can create many small files, increasing metadata operations and read overhead. Compaction through appropriate maintenance can improve scan behavior when file fragmentation is the actual problem. The correct maintenance interval depends on write volume, query patterns, available compute and the cost of the maintenance itself.<\/p>\n Fabric supports Delta maintenance<\/a> operations such as OPTIMIZE and VACUUM under supported settings. OPTIMIZE targets file layout and compaction; VACUUM removes unreferenced files according to retention constraints. They are not interchangeable. An aggressive VACUUM can interfere with time travel or recovery expectations, especially when retention is shortened beyond safe defaults.<\/p>\n Before scheduling maintenance, establish table ownership, consumer access windows, retention needs and whether the table is actively written by other jobs. A compacted table can still contain incorrect duplicate business rows. File optimization improves physical access; it does not replace quality validation or correct ingestion semantics.<\/p>\n Before running VACUUM, decide how long historical table versions and data files must remain available for replay, delayed consumers or audit. Optimizing a table does not delete logical rows in the way a source correction does; VACUUM can remove obsolete physical files needed by older snapshots after the retention boundary. A team promising thirty-day rollback must not choose a shorter physical retention merely because it saves storage. Test a representative time-travel or restore scenario in an isolated environment and reconcile the result with the approved recovery policy.<\/p>\n V-Order and related file-layout options can improve read behavior for appropriate Fabric consumers by changing storage organization or encoding. But they can introduce write overhead, and the optimal setting depends on the mix of readers and writers. Do not apply every available optimization simply because it has ‘fast’ in its product description. Observe the workloads and check the relevant engine’s supported defaults and control surface.<\/p>\n Partitioning can reduce scanned data when predicates reliably select a subset, but excessive high-cardinality partitions can create a larger small-file problem. A table partitioned per individual customer might produce thousands of tiny directories with little benefit for common queries. A stable, moderately selective time boundary may be more reasonable when query patterns align with it. Clustering or sorting decisions likewise depend on recurring filters and join keys.<\/p>\n A design test compares a representative query before and after one measured change while verifying identical results. If file compaction helps the query but makes frequent small updates too costly, record that tradeoff. A local speed result from a sample is not proof of the same improvement for all users or capacities.<\/p>\n Spark jobs can spend much of their time shuffling data across partitions for joins, grouping or sorting. Large shuffles, skewed business keys, repeated scans and expensive user-defined functions can dominate execution. Examine job plans and stage behavior rather than treating any long run as evidence that more workers are the only answer. Repartitioning can help one stage while increasing overhead elsewhere.<\/p>\n For example, joining a large transaction table to a small reference table may support a different execution strategy than joining two equally large fact tables. A single skewed customer ID can overload one partition even when the total cluster has spare capacity. Filter early when semantics permit, avoid unnecessary full-row materialization and verify that the chosen broadcast or repartition strategy is appropriate for observed sizes.<\/p>\nUnderstand Delta file sizing and table maintenance<\/h2>\n
V-Order, partitioning and clustering have tradeoffs<\/h2>\n
Spark performance is often about data movement<\/h2>\n