FACIL: Incremental Rule Learning from Data Streams

Transcripción

1 FCIL: Incremental Rule Learning from Data Streams Francisco J. Ferrer Troyano Jesús S. guilar, José C. Riquelme University of Sevilla Motivation Contents FCIL The hybrid knowledge model Three user parameters IIL updating The moderate growth distance Implicit and explicit forgetting heuristics The multi-strategy classification method Experiments Future work

2 Motivation Why Classify? Why Rules? Heterogeneous Data Sources Noise, Missing Values, Inconsistency High-Speed Data Fast algorithms, approximate answers Open-Ended Data On-line learning complex models? FCIL The hybrid knowledge model 210 Ritmo puls/min Primer enemigo recibido Frontera de Decisión e1 a1 migo obtenido para el 2º enemigo e2 a2 e3 a3 Edad Intervalos 12 Ritmo puls/min a1 a2 a3 Pres e2 a2 a3 e e1 e1 e3 e1 a1 e1 - e3 mm / Hg a1 - a Pres. 60 mm / Hg Ejemplos migos Ejemplos Enemigos Sop-Pos: 96 % Sop.Rel.=20% Prec.=98.5% order examples inside rules

3 FCIL The lgorithm Three user parameters Maximum number of rules / label necessary to bound the computational complexity Minimum accuracy (purity) / rule: p/(p+n) anytime,approximate answers Maximum growth / dimension FCIL IIL updating General IL pproach Successive Episodes Updating: M t = f(v t,m t-1 ) Window of examples V t : New [and] past examples IIL: Every example 1/3 cases: 1. Success 2. nomaly 3. Novelty

4 FCIL IIL / Success Éxito 0.5 a e a FCIL IIL / nomaly X e 3 a 3 a e 1 a 1 a 2 e

5 FCIL Learning pproach / Case 3 nomaly FCIL IIL / Novelty: The Growth Distance 0.9 d(r,x) = X d(r,x) = X R 0.4 R R 0.9 Selection: The rule implying the minimum growth C D

6 FCIL IIL / Novelty: The Moderate Growth Distance R X G(R a,x)= 0.2 G(R b,x)= R 0.6 Max. Dif. = 15% FCIL Implicit and Explicit Forgetting Heuristics ejemplos t=[17,33] t=35 e 16 e 34 e 16 e 16 reglas en t=16 reglas en t=34 e 16 e 16, e 29 e 34 último ejemplo e 34 e 29 e 28 e 19 e 19, e 28 multiple windows

7 FCIL Multi-strategy Classification??? Experiments General Purpose Classifier Evaluation FCIL as a UCI Repository 10 Folds Cross Validation, 10 times, t-student (0.05) Implicit Forgetting Heuristic Synthetic Data Streams Change Magnitude = ± for 40% attributes every 10 4 examples INPUT: 10 5 examples - 5% class noise: 900 training / 100 testing oth Implicit and Explicit Forgetting Heuristics

8 UCI Repository ccuracy 100 C4.5Rules FCIL Numerical ttributes alance reast C, Glass Heart S, Ionosphere Iris Diabetes Sonar Vehicle Vowel Waveform Wine UCI Repository ccuracy C4.5Rules FCIL Num. & Symbolic tts nneal utos Cleveland Credit Rating German Credit Hepatitis Horse Colic Zoo udoilogy reast Cancer Mushroom Primary Tumor Soybean Vote

9 UCI Repository Number of rules 90 C4.5Rules FCIL Numerical ttributes alance reast C, Glass Heart S, Ionosphere Iris Diabetes Sonar Vehicle Vowel Waveform Wine UCI Repository Number of rules C4.5Rules FCIL Num. & Simb. tts nneal utos Cleveland Credit Rating German Credit Hepatitis Horse Colic Zoo udoilogy reast Cancer Mushroom Primary Tumor Soybean Vote

10 UCI Repository Parametric Sensitivity 0,45 0,4 0,35 Time (seconds) 0,3 0,25 0,2 0,15 0,1 0, Valores límite para los tres parámetros En función del máximo número de reglas En función del máximo crecimiento permitido En función del umbral de mínima pureza UCI Repository Parametric Sensitivity 87 86, ,5 85 ccuracy (%) 84, , , , Valores límite para los tres parámetros En función del máximo número de reglas En función del máximo crecimiento permitido En función del umbral de mínima pureza

11 UCI Repository Parametric Sensitivity Number of rules Valores límite para los tres parámetros En función del máximo número de reglas En función del máximo crecimiento permitido En función del umbral de mínima pureza Moving Hyperplane ccuracy, time, and number of rules Máx. Growth = 50% Máx. Growth = 75% Nº ttributes ccuracy Time Nº Rules 25 ccuracy Time Nº Rules ,36 >3500 examples 14 / second >2500 examples 20 / second , , , ,88 >180 examples 277 / second >650 examples 77 / second10

12 Moving Hyperplane Sensitivity to the number of attributes Time Max Crec. = 50 %, Max Nº Reglas = 50 Max Crec. = 75 %, Max Nº Reglas = 100 Moving Hyperplane Sensitivity to the number of examples ccuracy Max Crec. = 75 %, Max Nº Reglas = 25 Max Crec. = 100 %, Max Nº Reglas = 10

13 Future Work Sensitivity to attribute Removing / Recovering attributes To adapt the Growth distance: Changing attribute weights improve the selection of rules Reordering attributes according influence speed-up the updating process Processing streams with a variable number of attributes To evaluate alternative distances between a rule and an example PreProcessing streams? Multiple Classification GRZIE MILLE Francisco Ferrer LSI - University of Seville ferrer@lsi.us.es