notes-shivam

syllabus

4 min read

1. Introduction to Data Warehousing

1.1 Data Warehousing

  • Definition of Data Warehousing

  • Characteristics of Data Warehouse

  • Purpose of Data Warehouse

1.2 Difference Between Operational Database System and Data Warehouse

  • Operational Database System

  • Data Warehouse

  • Comparison between OLTP and OLAP systems

1.3 Data Warehouse Users

  • Management Users

  • Analysts

  • Executives

  • Data Scientists and Researchers

1.4 Benefits of Data Warehousing

  • Improved decision making

  • Faster query response

  • Historical data analysis

  • Data consistency and integration

1.5 Metadata

  • Definition of Metadata

  • Role of Metadata in Data Warehouse

Classification of Metadata

  • Technical Metadata

  • Business Metadata

  • Operational Metadata

Importance of Metadata

  • Data understanding

  • Data management

  • Query optimization

  • Data integration support

1.6 Data Marts

  • Definition of Data Mart

  • Types of Data Marts

Reasons for Creating Data Marts

  • Department-specific analysis

  • Faster access to data

  • Reduced implementation cost

  • Improved performance

Building Data Marts

Top-Down Approach

  • Enterprise Data Warehouse first

  • Data marts created from warehouse

Bottom-Up Approach

  • Data marts created first

  • Combined later into warehouse

1.7 Data Warehouse Architecture

Two-Tier Architecture

  • Client layer

  • Data warehouse server layer

Three-Tier Architecture

  • Bottom tier

  • Middle tier

  • Top tier

1.8 Data Warehouse Schema

Star Schema

  • Fact table

  • Dimension tables

  • Characteristics and advantages

Snowflake Schema

  • Normalized dimension tables

  • Structure and features

Fact Constellation Schema

  • Multiple fact tables

  • Shared dimension tables

1.9 OLAP (Online Analytical Processing)

Need for OLAP

  • Multidimensional analysis

  • Decision support

  • Fast analytical queries

OLAP Operations

  • Roll-up

  • Drill-down

  • Slice

  • Dice

  • Pivot

OLAP Models

  • ROLAP

  • MOLAP

  • HOLAP

2. Data Preprocessing

2.1 Introduction to Data Preprocessing

  • Definition

  • Importance of preprocessing

2.2 Need for Data Preprocessing

  • Improve data quality

  • Handle incomplete data

  • Increase mining accuracy

2.3 Objectives of Data Preprocessing

  • Data cleaning

  • Data consistency

  • Reduction of redundancy

2.4 Techniques of Data Preprocessing

Descriptive Data Summarization

  • Statistical summaries

  • Visualization methods

Data Cleaning

  • Handling missing values

  • Removing noise

  • Correcting inconsistencies

Data Integration

  • Combining data from multiple sources

  • Schema integration

  • Entity identification

Data Transformation

  • Normalization

  • Aggregation

  • Generalization

Data Reduction

  • Data cube aggregation

  • Dimensionality reduction

  • Sampling

  • Compression

3. Introduction to Data Mining

3.1 Introduction to Data Mining

  • Definition

  • Evolution of data mining

3.2 Need for Data Mining

  • Extraction of useful knowledge

  • Pattern discovery

  • Business intelligence

3.3 KDD Process (Knowledge Discovery in Databases)

  • Data cleaning

  • Data integration

  • Data selection

  • Data transformation

  • Data mining

  • Pattern evaluation

  • Knowledge presentation

3.4 Data Mining Architecture

  • Database/Data warehouse server

  • Knowledge base

  • Data mining engine

  • Pattern evaluation module

  • User interface

3.5 Data Mining Functionalities

  • Concept description

  • Association analysis

  • Classification

  • Prediction

  • Clustering

  • Outlier analysis

3.6 Data Mining Task Primitives

  • Task-relevant data

  • Kind of knowledge to be mined

  • Background knowledge

  • Interestingness measures

3.7 Integration of Data Mining System with Database or Data Warehouse System

  • Coupling schemes

  • Benefits of integration

  • Performance improvement

4. Mining Frequent Items and Associations

4.1 Frequent Item Set

  • Definition

  • Support measure

4.2 Closed Item Set

  • Definition

  • Characteristics

4.3 Association Rule Mining

  • Definition

  • Rule generation

  • Support and confidence

4.4 Market Basket Analysis

  • Customer purchasing patterns

  • Product association analysis

4.5 Classification of Association Rules

  • Single-dimensional association rules

  • Multidimensional association rules

  • Boolean association rules

  • Quantitative association rules

4.6 Apriori Algorithm

  • Principle of Apriori

  • Candidate generation

  • Pruning

  • Advantages and limitations

5. Classification and Prediction

5.1 Classification and Prediction

  • Definition

  • Applications

5.2 Issues Regarding Classification and Prediction

  • Overfitting

  • Accuracy

  • Missing values

  • Scalability

5.3 Comparing Classification Methods

  • Accuracy

  • Speed

  • Robustness

  • Interpretability

5.4 Classification by Decision Tree Induction

  • Decision tree concept

  • Tree construction

  • Attribute selection

  • Advantages and disadvantages

6. Clustering

6.1 Introduction to Clustering

  • Definition

  • Characteristics

6.2 Cluster Analysis

  • Meaning

  • Objectives

6.3 Need for Clustering

  • Pattern recognition

  • Data segmentation

  • Knowledge discovery

6.4 Categorization of Major Clustering Methods

  • Partitioning methods

  • Hierarchical methods

  • Density-based methods

  • Grid-based methods

  • Model-based methods

6.5 Types of Data in Cluster Analysis

  • Interval-scaled variables

  • Binary variables

  • Nominal variables

  • Ordinal variables

  • Ratio-scaled variables

6.6 Partitioning Methods

K-Means Method

  • Working procedure

  • Advantages

  • Limitations

K-Medoids Method

  • Working procedure

  • Advantages

  • Limitations

6.7 Applications of Data Mining in Various Sectors

  • Banking

  • Healthcare

  • Education

  • Retail

  • Telecommunications

  • E-commerce

  • Fraud detection

  • Social media analytics

Graph View