Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining

Business Intelligence. Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining. Learning Objectives. Understand the concept and definitions of artificial neural networks (ANN) Know the similarities and differences between biological and artificial neural networks

Share Presentation
Embed Code
Link
Download Presentation

zuwena

zuwena + Follow

Download Presentation

Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

Presentation Transcript

  1. Business Intelligence Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining Business Intelligence
  2. Learning Objectives • Understand the concept and definitions of artificial neural networks (ANN) • Know the similarities and differences between biological and artificial neural networks • Learn the different types of neural network architectures • Learn the advantages and limitations of ANN • Understand how backpropagation learning works in feedforward neural networks Business Intelligence
  3. Learning Objectives • Understand the step-by-step process of how to use neural networks • Appreciate the wide variety of applications of neural networks; solving problem types of • Classification • Regression • Clustering • Association • Optimization Business Intelligence
  4. Opening Vignette:(Page 242) “Predicting Gambling Referenda with Neural Networks” • Using NeuroSolutions, this study developed and tested models to predict community support for commercial gaming. • The study examined the role of factors that contribute to legalization and/or probation of gambling activities using neural networks. • It attempted to use Neural Network technology to predict various counties voting outcome on this subject. Business Intelligence
  5. Opening Vignette: • On average, the models accurately predicted the voting results for 4 out of every 5 counties (approximately 82% accuracy) on a sample data set. (1287 records of data) • Interestingly, and contrary to popular belief, the counties financial characteristics and age distribution were not found to be significant factors in determining ballot outcome. Dominant factors are identified on Page 244 • The study demonstrates that demographic data can be used to accurately predict voting outcomes on controversial issues. Business Intelligence
  6. Opening Vignette:Predicting Gambling Referenda… Business Intelligence
  7. Opening Vignette: • NeuroSolutions is offered by NeuroDimension and offers algorithms for the in the field of artificial intelligence. • NeuroDimension offers NeuroSolutions, NeuroSolutions for Excel, and a Custom Solution Wizard each of which can be downloaded for a free eval. Business Intelligence
  8. Opening Vignette: • An very good video is offered by the company that explains Neural Network algorithms and the field in general. • Pricing is relatively reasonable for the product. - NS for Excel costs $295 Business Intelligence
  9. Neural Network Concepts(Page 245) • Neural networks (NN): a brain metaphor for information processing– uses artificial neurons (programming constructs that mimic the properties of biological neurons). • Neural computing - pattern recognition methodology for machine learning • Artificial neural network (ANN) – resulting model from neural computing • Many uses for ANN for • pattern recognition, forecasting, prediction, and classification • finance, marketing, manufacturing, operations, information systems, and so on Business Intelligence
  10. ANN Video • Here is an excellent video offered by NeuroSolutions that provides a good overview of ANN Business Intelligence
  11. Biological Neural Networks(Page 246) • Two interconnected brain cells (neurons) • An axon is a long, slender projection of a nerve cell, or neuron, that conducts electrical impulses away from the neuron's cell body or soma. • Dendrites are branched filaments in nerve cells (neurons). The word dendrite derives from the Greek word for tree which describes their branching tree-like structure. Business Intelligence
  12. Biological Neural Networks(Page 246) • Synapse – able to increase or decrease the strength of the connection between neurons and cause excitation or inhibition of a subsequent neuron. • The word "soma" comes from the the Greek word “body”; the soma of a neuron is often called the cell body. Business Intelligence
  13. Processing Information in ANN(Page 247 Figure 6.3) • A single neuron (processing element – PE) with inputs and outputs Business Intelligence
  14. Biology Analogy(Page 247) Business Intelligence
  15. Elements of ANN(Page 248-250) • Processing element (PE) – organized in different ways to form the networks structure. • Network architecture • Hidden layers - takes input from the previous layer and converts into outputs for more processing (used in complex problems) • Parallel processing – resembles the way the brain works – different than serial processing in conventional computing Not this ANN Business Intelligence
  16. Elements of ANN(Page 248-250) • Network information processing • Inputs – single attribute such as age, income level, etc • Outputs – solution to the problem – ie – loan app “yes” or “no” • Connection weights – relative strength of input data (how important) • Summation function – weighted sums of all input elements entering a PE. Business Intelligence
  17. Elements of ANN(Figure 6.4 Page 249) Neural Network with One Hidden Layer Business Intelligence
  18. Elements of ANN Summation Function for a Single Neuron (a) and Several Neurons (b) Business Intelligence
  19. Elements of ANN(Page 251) • Transformation (Transfer) Function – activation level of a neuron (based on this level the neuron may or may not produce an output). • Computed via Sigmoid (logical activation) function – YT=1/(1+e-Y) • Y is computed via weighted summation • Any value less than threshold will not be passed to output (0); anything above does (1) • Threshold value Business Intelligence
  20. Neural Network Architectures(Page 251-252) • Several ANN architectures exist • Feedforward - figure 6.4 page 249 (see previous slide) • Recurrent - - figure 6.7 page 252 (next slide) • Associative memory • Self-organizing feature maps • Hopfield networks, etc Business Intelligence
  21. Neural Network ArchitecturesRecurrent Neural Networks(Page 252, figure 6.7) Business Intelligence
  22. Neural Network Architectures(Page 252) • Architecture of a neural network is driven by the task it is intended to address • Most popular architecture: Feedforward, multi-layered perceptron with backpropagation learning algorithm • Ie – Feedforward Perceptron is the architecture and backpropagation is the learning algorithm. Business Intelligence
  23. Neural Network Architectures • The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. Frank Rosenblatt was a computer scientist born in 1928 in New York City. He helped to create the Perceptron, a.k.a. the Mark 1, computer in 1960 at Cornell University. This was the first computer that could learn skills by trial and error in an attempt to mimic human thought processes through the use of a neural network. (Died 1971) • Backpropagation is a common, supervised method for teaching artificial neural networks how to perform a given task. It was first described by Arthur E. Bryson and Yu-Chi Ho in 1969. Frank Rosenblatt (1957) Business Intelligence
  24. Neural Network Architectures Original Mark 1 (Automatic Sequence Controlled Calculator (ASCC) The building elements of the ASCC were switches, relays, rotating shafts, and clutches. Business Intelligence
  25. Learning in ANN(Page 252) • A process by which a neural network learns the underlying relationship between input and outputs, or just among the inputs • Supervised learning • For prediction type problems • E.g., backpropagation • Unsupervised learning • For clustering type problems • Self-organizing • E.g., adaptive resonance theory Business Intelligence
  26. A Taxonomy of ANN Learning Algorithms(Page 253, Figure 6.8) Most popular Business Intelligence
  27. Read Application Case(Page 254) • Microsoft used BrainMaker Neural Network software from California Scientific to maximize return on direct Mail • Some of the variables considered (25 in total) • Recency (how long since last registration / product purchase) • First date to file – loyal over time? • Number of products bought and filed • Value of products bought and registered • Number of days from product release to purchase • Improved response rate from 4.9% to 8.2% - 35% cost savings on 40 Millions pieces of direct mailings Business Intelligence
  28. A Supervised Learning Process(Pages 255-256, figure 6.9) Three-step process: 1. Compute temporary outputs 2. Compare outputs with desired targets 3. Adjust the weights and repeat the process Business Intelligence
  29. How a Network Learns(Page 256) • Example: single neuron that learns the inclusive OR operation • Learning parameters: • Learning rate • Momentum * See page 257 for step-by-step progression of the learning process Business Intelligence
  30. Backpropagation Learning(Page 258) • Errors are used to correct weights – called Back-error propagation • The (supervised) learning algorithm procedure: • Initialize weights with random values and set other network parameters • Read in the inputs and the desired outputs • Compute the actual output (by working forward through the layers) • Compute the error (difference between the actual and desired output) • Change the weights by working backward through the hidden layers • Repeat steps 2-5 until weights stabilize Business Intelligence
  31. Backpropagation Learning(Figure 6.10 Page 258) • Backpropagation of Error for a Single Neuron Business Intelligence
  32. Development Process of an ANN(Page 259) Similar to structured design for traditional IS, with some new elements See page 253 Business Intelligence
  33. An MLP ANN Structure for the Box-Office Prediction Problem (Page 262, Fig 6.12)This is the vignette at the start of Chapter 5 of page 191 Business Intelligence
  34. Data Collection and Testing(Page 261) • Data is split into three parts • Training (~60%) • Validation (~20%) • Testing (~20%) Business Intelligence
  35. Sensitivity Analysis on ANN Models(Page 264-265) • A common criticism for ANN: The black-box syndrome! • Answer: sensitivity analysis • Conducted on a trained ANN • The inputs are changed while the relative change on the output is measured/recorded • Results illustrates the relative importance of input variables Business Intelligence
  36. Sensitivity Analysis on ANN Models(Page 265, Figure 6.13) • See and read example Application Case 6.5 (Page 266) • Sensitivity analysis reveals the most important injury severity factors in traffic accidents Business Intelligence
  37. Sensitivity Analysis on ANN Models(Page 266) • Application Case 6.5 – see here • 41,000 die in 6M US traffic accidents • Analyze the factors that elevate the risk of severe injury • Factors include behavior, environment, technical, etc. • Used series of ANN models to estimate the significance of the crash factors on the level of severity sustained by the driver. • Two step process used (1) prediction models, (2) sensitivity analysis on trained neural network • Results shows significant differences among models built for different injury severity levels. (The most influential factors HIGHLY depend on the level of injury). Business Intelligence
  38. A Sample Neural Network ProjectBankruptcy Prediction(Pg 267-270) • A comparative analysis of ANN versus logistic regression (LR) (a statistical method) • Inputs • X1: Working capital/total assets • X2: Retained earnings/total assets • X3: Earnings before interest and taxes/total assets • X4: Market value of equity/total debt • X5: Sales/total assets Business Intelligence
  39. A Sample Neural Network ProjectBankruptcy Prediction • Data was obtained from Moody's Industrial Manuals • Time period: 1975 to 1982 • 129 firms (65 of which went bankrupt during the period and 64 nonbankrupt) • Different training and testing propositions are used/compared • 90/10 versus 80/20 versus 50/50 • Resampling is used to create 60 data sets Business Intelligence
  40. A Sample Neural Network ProjectBankruptcy Prediction • Network Specifics • Feedforward MLP • Backpropagation • Varying learning and momentum values • 5 input neurons (1 for each financial ratio), • 10 hidden neurons, • 2 output neurons (1 indicating a bankrupt firm and the other indicating a nonbankrupt firm) Business Intelligence
  41. A Sample Neural Network ProjectBankruptcy Prediction – Results(Page 269 figure 6.2) Business Intelligence
  42. Bottomline - Advantages of ANN(Pages 274-276) • Able to deal with (identify/model) highly nonlinear relationships • Can handle variety of problem types (loan apps, forecast profitability / finances, sports – team success, fraud prevention, time-series forecasting, health care and medicine – diagnose breast cancer – see Case 6.4 on page 276) • Usually provides better results (prediction and/or clustering) compared to its statistical counterparts Business Intelligence
  43. Disadvantages of ANN • They are deemed to be black-box solutions, lacking expandability • It is hard to find optimal values for large number of network parameters • Optimal design is still an art: requires expertise and extensive experimentation • It is hard to handle large number of variables (especially the rich nominal attributes) • Training may take a long time for large datasets; which may require case sampling Business Intelligence
  44. ANN Software(Page 263) • Standalone ANN software tool • NeuroSolutions • BrainMaker • NeuralWare • NeuroShell, … for more (see pcai.com) … • Part of a data mining software suit • PASW (formerly SPSS Clementine) • SAS Enterprise Miner • Statistica Data Miner, … many more … Business Intelligence
  45. Next lecture Chapter 7 - Text and Web Mining Business Intelligence

Artificial Neural Networks

Artificial Neural Networks

Articifial Intelligence. Artificial Neural Networks. Brian Talecki CSC 8520 Villanova University. ANN - Artificial Neural Network. A set of algebraic equations and functions which determine the best output given a set of inputs.

1.06k views • 41 slides

Data Mining - CSE5230

Data Mining - CSE5230

CSE5230/DMS/2004/7. Data Mining - CSE5230. Classifiers 3 Feedforward Neural Networks. Lecture Outline. Why study neural networks? What are neural networks and how do they work? History of artificial neural networks (ANNs) Applications and advantages Choosing and preparing data

732 views • 35 slides

Introduction to Business Analytics

Introduction to Business Analytics

Introduction to Business Analytics. Chapter 6: Neural Networks for Data Mining Matthew J. Liberatore Thomas Coghlan Fall 2008. Learning Objectives. Understand the concept and different types of artificial neural networks (ANN) Learn the advantages and limitations of ANN

862 views • 39 slides

Data Mining and Neural Networks

Data Mining and Neural Networks

2. Artificial Intelligence for Data Mining. Neural networks are useful for data mining and decision-support applications.People are good at generalizing from experience.Computers excel at following explicit instructions over and over. Neural networks bridge this gap by modeling, on a computer, the neural behavior of human brains..

606 views • 16 slides

<a href=CSE 592 Applications of Artificial Intelligence Neural Networks & Data Mining" width="320px" />

CSE 592 Applications of Artificial Intelligence Neural Networks & Data Mining

CSE 592 Applications of Artificial Intelligence Neural Networks & Data Mining. Henry Kautz Winter 2003. Kinds of Networks. Feed-forward Single layer Multi-layer Recurrent. Kinds of Networks. Feed-forward Single layer Multi-layer Recurrent. Kinds of Networks. Feed-forward

1.26k views • 105 slides

Neural Networks

Neural Networks

Neural Networks. MSE 2400 EaLiCaRA Spring 2014 Dr. Tom Way. Background. Neural Networks can be : - Biological models - Artificial models Desire to produce artificial systems capable of sophisticated computations similar to the human brain .

583 views • 35 slides

LECTURE 16: NEURAL NETWORKS

LECTURE 16: NEURAL NETWORKS

LECTURE 16: NEURAL NETWORKS. Objectives: Feedforward Networks Multilayer Networks Backpropagation Posteriors Kernels Resources: DHS: Chapter 6 AM: Neural Network Tutorial NSFC: Introduction to NNs GH: Short Courses. Overview.

380 views • 15 slides

Lecture 5: Automatic cluster detection Lecture 6: Artificial neural networks

Lecture 5: Automatic cluster detection Lecture 6: Artificial neural networks

Brief introduction to lectures. Lecture 5: Automatic cluster detection Lecture 6: Artificial neural networks Lecture 7: Evaluation of discovered knowledge. Transparencies prepared by Ho Tu Bao [JAIST]. Lecture 5: Automatic Cluster Detection .

347 views • 16 slides

Artificial Intelligence Chapter 3 Neural Networks

Artificial Intelligence Chapter 3 Neural Networks

Artificial Intelligence Chapter 3 Neural Networks. Biointelligence Lab School of Computer Sci. & Eng. Seoul National University. Outline. 3.1 Introduction 3.2 Training Single TLUs Gradient Descent Widrow-Hoff Rule Generalized Delta Procedure 3.3 Neural Networks

425 views • 25 slides

Neural Networks

Neural Networks

Neural Networks. Automatic Model Building (Machine Learning) Artificial Intelligence. High-Growth Product. Used for classifying data target customers bank loan approval hiring stock purchase trading electricity DATA MINING Used for prediction. Description.

303 views • 18 slides

An Introduction to Artificial Neural Network Models

An Introduction to Artificial Neural Network Models

An Introduction to Artificial Neural Network Models. John Meech, Norman B. Keevil Institute of Mining Engineering, The University of British Columbia. What are Artificial Neural Networks ?. Artificial Neural Networks: A class of models that mimic the functioning of the human brain.

731 views • 25 slides

Artificial Neural Networks

Artificial Neural Networks

Artificial Neural Networks. Dr. Lahouari Ghouti Information & Computer Science Department. Review of Probability Concepts. Why Probabilities. The world is a very uncertain place 30 years of Artificial Intelligence, Machine Learning and Data-mining research evolved around this fact!

520 views • 31 slides

Lecture 7

Lecture 7

Lecture 7. Artificial neural networks: Supervised learning. Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in multilayer neural networks The Hopfield network

924 views • 73 slides

Chapter 6

Chapter 6

Chapter 6. Artificial neural networks:. Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks. Neural Networks and the Brain. A neural network is a model of reasoning inspired by the human brain.

688 views • 49 slides

Lecture 7

Lecture 7

Lecture 7. Artificial neural networks: Supervised learning. Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in multilayer neural networks The Hopfield network

898 views • 73 slides

Data Mining with Neural Networks

Data Mining with Neural Networks

Data Mining with Neural Networks. Standard data mining terminology Preprocessing data Running neural networks via Analyze/StripMiner Cherkassky’s nonlinear regression problem Magnetocardiogram data CBA (chemical and biological agents) Data Drug design with neural networks

600 views • 37 slides

Artificial Neural Networks

Artificial Neural Networks

Artificial Neural Networks. Kong Da, Xueyu Lei & Paul McKay. Neural Networks. What is Neural Networks. An Artificial Neural Network is a computational simulation of a biological neural network. composed of a large number of highly interconnected processing elements(neurons )

876 views • 17 slides

Introduction to Artificial Intelligence (G51IAI)

Introduction to Artificial Intelligence (G51IAI)

Introduction to Artificial Intelligence (G51IAI). Dr Matthew Hyde Neural Networks. More precisely: “ Artificial Neural Networks” Simulating, on a computer, what we understand about neural networks in the brain. Lecture Outline. Recap on perceptrons Linear Separability

604 views • 37 slides

INTRODUCTION TO ARTIFICIAL INTELLIGENCE

INTRODUCTION TO ARTIFICIAL INTELLIGENCE

INTRODUCTION TO ARTIFICIAL INTELLIGENCE. Massimo Poesio LECTURE 15: Artificial Neural Networks. ARTIFICIAL NEURAL NETWORKS. Analogy to biological neural systems, the most robust learning systems we know. Attempt to understand natural biological systems through computational modeling.

598 views • 38 slides

Chapter 8

Chapter 8

Chapter 8 . NEURAL NETWORKS FOR DATA MINING. Learning Objectives. Understand the concept and different types of artificial neural networks (ANN) Learn the advantages and limitations of ANN Understand how backpropagation neural networks learn

580 views • 43 slides

Chapter 7 Neural <a href=Networks in Data Mining" width="320px" />

Chapter 7 Neural Networks in Data Mining

Chapter 7 Neural Networks in Data Mining. Automatic Model Building (Machine Learning) Artificial Intelligence. Contents. Describe neural networks as used in Data mining Reviews real applications of each model Shows the application of models to larger data sets. High-Growth Product.

1.26k views • 18 slides