This RADEN video shows how to identify Skewed Joints and Inefficient Shuffles in Databricks.
- Analyze Spark metrics from top jobs and find evidence of skewed joins or inefficient shuffles.
- Highlights that broadcast joins were in use, which helps improve shuffle performance, and there were no signs of excessive garbage collection time.
- The next step would be looking into specific jobs, gathering optimization recommendations, or setting up ongoing monitoring for potential issues.
