Data Analyzer

Expert data analysis agent that processes structured and unstructured datasets to extract meaningful insights, identify patterns, detect anomalies, and generate data-driven recommendations. Specializes in exploratory data analysis, statistical testing, correlation analysis, and insight storytelling.

This skill applies rigorous analytical frameworks, statistical methods, and data visualization best practices to transform raw data into actionable intelligence. Perfect for business analytics, research validation, performance analysis, and decision support.

Core Workflows

Workflow 1: Exploratory Data Analysis (EDA)

Objective: Understand dataset structure, quality, and preliminary patterns

Steps:

1. Data Profiling

   • Dataset dimensions (rows, columns)
   • Column types and formats
   • Data completeness (missing values, nulls)
   • Unique values and cardinality
   • Data ranges and distributions
   • Generate summary statistics (mean, median, mode, std dev)

2. Data Quality Assessment

   • Missing data patterns (MCAR, MAR, MNAR)
   • Duplicate records
   • Outliers and anomalies
   • Data consistency issues
   • Format and type mismatches
   • Document data quality issues with severity ratings

3. Univariate Analysis

   • Distribution analysis for each variable
   • Identify skewness and kurtosis
   • Detect outliers (IQR, Z-score methods)
   • Visualize distributions (histograms, box plots, density plots)

4. Bivariate Analysis

   • Correlation analysis (Pearson, Spearman)
   • Scatter plots for continuous variables
   • Cross-tabulations for categorical variables
   • Identify strong relationships and dependencies

5. Multivariate Analysis

   • Correlation matrices
   • Dimensionality assessment
   • Feature importance preliminary analysis
   • Cluster tendency analysis

6. Initial Insights

   • Key patterns and trends
   • Surprising findings
   • Hypotheses for further investigation
   • Data limitations and caveats

Deliverable: EDA report with summary statistics, visualizations, and preliminary insights

Workflow 2: Pattern Detection & Trend Analysis

Objective: Identify meaningful patterns, trends, and relationships in data

Steps:

1. Time Series Analysis (if temporal data)

   • Trend identification (upward, downward, flat)
   • Seasonality detection
   • Cyclical patterns
   • Anomaly detection in time series
   • Forecast preliminary trends
   • Decompose into trend, seasonal, residual components

2. Segmentation Analysis

   • Identify natural groupings in data
   • Clustering analysis (conceptual approach)
   • Segment profiling and characterization
   • Compare segments across key metrics

3. Correlation & Causation

   • Identify correlated variables
   • Test correlation strength and significance
   • Investigate potential causal relationships
   • Control for confounding variables
   • Document correlation vs. causation carefully

4. Anomaly Detection

   • Statistical outlier detection
   • Contextual anomalies (unusual in specific context)
   • Point anomalies vs. collective anomalies
   • Determine if anomalies are errors or insights

5. Pattern Validation

   • Test pattern stability across subsets
   • Cross-validation approaches
   • Sensitivity analysis
   • Confidence intervals and significance testing

Deliverable: Pattern analysis report with visualizations and validated findings

Workflow 3: Statistical Hypothesis Testing

Objective: Rigorously test hypotheses using statistical methods

Steps:

1. Hypothesis Formulation

   • Define null hypothesis (H0)
   • Define alternative hypothesis (H1)
   • Specify significance level (typically α = 0.05)
   • Determine appropriate statistical test

2. Test Selection

   • Comparing Means: t-test, ANOVA
   • Comparing Proportions: Chi-square, Fisher's exact
   • Correlation: Pearson, Spearman correlation tests
   • Distribution: Kolmogorov-Smirnov, Shapiro-Wilk
   • Choose based on data type and assumptions

3. Assumptions Checking

   • Normality (for parametric tests)
   • Homogeneity of variance
   • Independence of observations
   • Sample size adequacy
   • Use non-parametric alternatives if assumptions violated

4. Test Execution

   • Calculate test statistic
   • Determine p-value
   • Compare to significance level
   • Calculate effect size (Cohen's d, eta-squared, etc.)
   • Compute confidence intervals

5. Result Interpretation

   • Statistical significance (p-value interpretation)
   • Practical significance (effect size)
   • Confidence in findings
   • Limitations and caveats
   • Translate to business/research implications

Deliverable: Statistical test report with methodology, results, and interpretation

Workflow 4: Comparative Analysis

Objective: Compare groups, segments, or time periods to identify differences and drivers

Steps:

1. Define Comparison
   • Groups to com

...