Knowledge Discovery in Databases (KDD)

Knowledge Discovery in Databases (KDD) KDD is an iterative process where evaluation measures can be enhanced, mining can be refined, new data can be integrated and transformed in order to get different and more appropriate results.

Steps in Knowledge Discovery in Databases (KDD):

Some people treat data mining same as Knowledge discovery while some people view data mining essential step in process of knowledge discovery. Here is the list of steps involved in knowledge discovery process:

Data Cleaning – In this step the noise and inconsistent data is removed.
Data Integration – In this step multiple data sources are combined.
Data Selection – In this step relevant to the analysis task are retrieved from the database.
Data Transformation – In this step data are transformed or consolidated into forms appropriate for mining by performing summary or aggregation operations.
Data Mining – In this step intelligent methods are applied in order to extract data patterns.
Pattern Evaluation – In this step, data patterns are evaluated.
Knowledge Presentation – In this step, knowledge is represented.

Meta Data Repository

Metadata are data about data. When used in a data warehouse, metadata are the data that define warehouse objects. Metadata are created for the data names and definitions of the given warehouse.

Additional metadata are created and captured for time stamping any extracted data, the source of the extracted data, and missing fields that have been added by data cleaning or integration processes.

Characteristics of Meta Data Repository

The algorithms used for summarization, which include measure and dimension definitionalgorithms, data on granularity, partitions, subject areas, aggregation, summarization,and predefined queries and reports.
The mapping from the operational environment to the data warehouse, which includessource databases and their contents, gateway descriptions, data partitions, data extraction, cleaning, transformation rules and defaults, data refresh and purging rules, andsecurity (user authorization and access control).
Data related to system performance, which include indices and profiles that improvedata access and retrieval performance, in addition to rules for the timing and scheduling of refresh, update, and replication cycles.
Business metadata, which include business terms and definitions, data ownership information, and charging policies.

Summary

KDD refers to the overall process of discovering useful knowledge from data. It involves the evaluation and possibly interpretation of the patterns to make the decision of what qualifies as knowledge.