Authors:

V.S. Siyou Fotso E. Mephu Nguifo P. Vaslin
Email: siyou@isima.fr Email: mephu@isima.fr Email: vaslin@isima.fr
LIMOS, UMR 6158 LIMOS, UMR 6158 LIMOS, UMR 6158
Clemont Auvergne University , BP 10125, Clemont Auvergne University , BP 10125, Clemont Auvergne University , BP 10125,
63173, Clermont Ferrand, France 63173, Clermont Ferrand, France 63173, Clermont Ferrand, France

Datasets:

General description

We have performed an extensive set of experiments to evaluate the effectiveness and efficiency of our approach on a 84 different real and synthetic datasets (all the datasets of UCR except ElectricDevice).

Results:

We implemented our approach in R and C++ languages. All the experiments of our approach were carried out using a local cluster with 32 nodes. Subsequently, we present a comparison of the method we propose to that of the literature. The training set is used to find a segment number which is then used for the time series classification of the testing set with 1NN and PDTW.

The methods are compared using the Wilcoxon statistical test. A p-value less than 0.05 shows that :

Comparison of the accuracy of IDDTW and FDTW on 84 datasets

Summary table of comparisons of classification algorithms with Wilconox test

FDTW > < =
NB DTW_F SVMQ
C45 ST RANDF
Logistic BOSS ROTF
BN EE MLP
COTE EUCLIDEAN_1_NN
DDTW_R1_1NN
DDTW_RN_1NN
ERP_1NN
LCSS_1NN
MSM_1NN
TWE_1NN
WDDTW_1NN
WDTW_1NN
DD_DTW
DTD_C
LS
BOP
SAXVSM
TSF
TSBF
LPS
PS
CID_DTW
SVML
FS
ACF

NB and FDTW comparison

wilcox.test(NB, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  NB and FDTW
## W = 2423, p-value = 0.0004586
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, NB, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and NB
## W = 4633, p-value = 0.0002293
## alternative hypothesis: true location shift is greater than 0

C45 and FDTW comparison

wilcox.test(C45, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  C45 and FDTW
## W = 2144, p-value = 1.139e-05
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, C45, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and C45
## W = 4912, p-value = 5.696e-06
## alternative hypothesis: true location shift is greater than 0

SVML and FDTW comparison

wilcox.test(SVML, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  SVML and FDTW
## W = 3013, p-value = 0.1026
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, SVML, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and SVML
## W = 4043, p-value = 0.05132
## alternative hypothesis: true location shift is greater than 0

SVMQ and FDTW comparison

wilcox.test(SVMQ, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  SVMQ and FDTW
## W = 3455.5, p-value = 0.8193
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, SVMQ, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and SVMQ
## W = 3600.5, p-value = 0.4097
## alternative hypothesis: true location shift is greater than 0

Logistic and FDTW comparison

wilcox.test(LOGISTIC, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  LOGISTIC and FDTW
## W = 2313.5, p-value = 0.0001758
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, LOGISTIC, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and LOGISTIC
## W = 4658.5, p-value = 8.792e-05
## alternative hypothesis: true location shift is greater than 0

BN and FDTW comparison

wilcox.test(BN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  BN and FDTW
## W = 2625, p-value = 0.004196
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, BN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and BN
## W = 4431, p-value = 0.002098
## alternative hypothesis: true location shift is greater than 0

RANDF and FDTW comparison

wilcox.test(RANDF, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  RANDF and FDTW
## W = 3448, p-value = 0.8009
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, RANDF, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and RANDF
## W = 3608, p-value = 0.4004
## alternative hypothesis: true location shift is greater than 0

ROTF and FDTW comparison

wilcox.test(ROTF, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  ROTF and FDTW
## W = 3748.5, p-value = 0.4852
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, ROTF, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and ROTF
## W = 3307.5, p-value = 0.7584
## alternative hypothesis: true location shift is greater than 0

MLP and FDTW comparison

wilcox.test(MLP, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  MLP and FDTW
## W = 3290, p-value = 0.4512
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, MLP, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and MLP
## W = 3766, p-value = 0.2256
## alternative hypothesis: true location shift is greater than 0

EUCLIDEAN_1_NN and FDTW comparison

wilcox.test(EUCLIDEAN_1_NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  EUCLIDEAN_1_NN and FDTW
## W = 3122.5, p-value = 0.1989
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, EUCLIDEAN_1_NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and EUCLIDEAN_1_NN
## W = 3933.5, p-value = 0.09943
## alternative hypothesis: true location shift is greater than 0

DDTW_R1_1NN and FDTW comparison

wilcox.test(DDTW_R1_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  DDTW_R1_1NN and FDTW
## W = 3141, p-value = 0.2202
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, DDTW_R1_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and DDTW_R1_1NN
## W = 3915, p-value = 0.1101
## alternative hypothesis: true location shift is greater than 0

DDTW_RN_1NN and FDTW comparison

wilcox.test(DDTW_RN_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  DDTW_RN_1NN and FDTW
## W = 3440.5, p-value = 0.7826
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, DDTW_RN_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and DDTW_RN_1NN
## W = 3615.5, p-value = 0.3913
## alternative hypothesis: true location shift is greater than 0

ERP_1NN and FDTW comparison

wilcox.test(ERP_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  ERP_1NN and FDTW
## W = 3732.5, p-value = 0.5175
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, ERP_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and ERP_1NN
## W = 3323.5, p-value = 0.7423
## alternative hypothesis: true location shift is greater than 0

LCSS_1NN and FDTW comparison

wilcox.test(LCSS_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  LCSS_1NN and FDTW
## W = 3680, p-value = 0.6308
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, LCSS_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and LCSS_1NN
## W = 3376, p-value = 0.6857
## alternative hypothesis: true location shift is greater than 0

MSM_1NN and FDTW comparison

wilcox.test(MSM_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  MSM_1NN and FDTW
## W = 3986, p-value = 0.1467
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, MSM_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and MSM_1NN
## W = 3070, p-value = 0.9271
## alternative hypothesis: true location shift is greater than 0

TWE_1NN and FDTW comparison

wilcox.test(TWE_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  TWE_1NN and FDTW
## W = 3784.5, p-value = 0.4167
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, TWE_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and TWE_1NN
## W = 3271.5, p-value = 0.7925
## alternative hypothesis: true location shift is greater than 0

comparison WDDTW_1NN and FDTW

wilcox.test(WDDTW_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  WDDTW_1NN and FDTW
## W = 3446.5, p-value = 0.7972
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, WDDTW_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and WDDTW_1NN
## W = 3609.5, p-value = 0.3986
## alternative hypothesis: true location shift is greater than 0

WDTW_1NN and FDTW comparison

wilcox.test(WDTW_1NN, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  WDTW_1NN and FDTW
## W = 3787.5, p-value = 0.4113
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, WDTW_1NN, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and WDTW_1NN
## W = 3268.5, p-value = 0.7953
## alternative hypothesis: true location shift is greater than 0

DD_DTW and FDTW comparison

wilcox.test(DD_DTW, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  DD_DTW and FDTW
## W = 3787, p-value = 0.4122
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, DD_DTW, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and DD_DTW
## W = 3269, p-value = 0.7948
## alternative hypothesis: true location shift is greater than 0

DTD_C and FDTW comparison

wilcox.test(DTD_C, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  DTD_C and FDTW
## W = 3843, p-value = 0.3184
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, DTD_C, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and DTD_C
## W = 3213, p-value = 0.8416
## alternative hypothesis: true location shift is greater than 0

DTW_F and FDTW comparison

wilcox.test(DTW_F, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  DTW_F and FDTW
## W = 4140, p-value = 0.0524
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, DTW_F, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and DTW_F
## W = 2916, p-value = 0.974
## alternative hypothesis: true location shift is greater than 0

ST and FDTW comparison

wilcox.test(ST, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  ST and FDTW
## W = 4539, p-value = 0.001348
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, ST, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and ST
## W = 2517, p-value = 0.9993
## alternative hypothesis: true location shift is greater than 0

LS and FDTW comparison

wilcox.test(LS, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  LS and FDTW
## W = 3920, p-value = 0.2143
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, LS, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and LS
## W = 3136, p-value = 0.8935
## alternative hypothesis: true location shift is greater than 0

FS and FDTW comparison

wilcox.test(FS, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FS and FDTW
## W = 3040.5, p-value = 0.1224
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, FS, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and FS
## W = 4015.5, p-value = 0.06118
## alternative hypothesis: true location shift is greater than 0

BOP and FDTW comparison

wilcox.test(BOP, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  BOP and FDTW
## W = 3374.5, p-value = 0.6274
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, BOP, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and BOP
## W = 3681.5, p-value = 0.3137
## alternative hypothesis: true location shift is greater than 0

SAXVSM and FDTW comparison

wilcox.test(SAXVSM, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  SAXVSM and FDTW
## W = 3412, p-value = 0.7141
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, SAXVSM, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and SAXVSM
## W = 3644, p-value = 0.357
## alternative hypothesis: true location shift is greater than 0

BOSS and FDTW comparison

wilcox.test(BOSS, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  BOSS and FDTW
## W = 4508.5, p-value = 0.001878
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, BOSS, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and BOSS
## W = 2547.5, p-value = 0.9991
## alternative hypothesis: true location shift is greater than 0

TSF and FDTW comparison

wilcox.test(TSF, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  TSF and FDTW
## W = 3963, p-value = 0.1681
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, TSF, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and TSF
## W = 3093, p-value = 0.9164
## alternative hypothesis: true location shift is greater than 0

TSBF and FDTW comparison

wilcox.test(TSBF, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  TSBF and FDTW
## W = 4043, p-value = 0.1026
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, TSBF, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and TSBF
## W = 3013, p-value = 0.949
## alternative hypothesis: true location shift is greater than 0

LPS and FDTW comparison

wilcox.test(LPS, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  LPS and FDTW
## W = 3986.5, p-value = 0.1462
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, LPS, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and LPS
## W = 3069.5, p-value = 0.9273
## alternative hypothesis: true location shift is greater than 0

ACF and FDTW comparison

wilcox.test(ACF, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  ACF and FDTW
## W = 3041.5, p-value = 0.1231
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, ACF, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and ACF
## W = 4014.5, p-value = 0.06157
## alternative hypothesis: true location shift is greater than 0

PS and FDTW comparison

wilcox.test(PS, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  PS and FDTW
## W = 3289, p-value = 0.4493
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, PS, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and PS
## W = 3767, p-value = 0.2246
## alternative hypothesis: true location shift is greater than 0

EE and FDTW comparison

wilcox.test(EE, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  EE and FDTW
## W = 4231.5, p-value = 0.02573
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, EE, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and EE
## W = 2824.5, p-value = 0.9872
## alternative hypothesis: true location shift is greater than 0

COTE and FDTW comparison

wilcox.test(COTE, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  COTE and FDTW
## W = 4809, p-value = 4.861e-05
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, COTE, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and COTE
## W = 2247, p-value = 1
## alternative hypothesis: true location shift is greater than 0

CID_DTW and FDTW comparison

wilcox.test(CID_DTW, FDTW)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  CID_DTW and FDTW
## W = 3812, p-value = 0.3685
## alternative hypothesis: true location shift is not equal to 0
wilcox.test(FDTW, CID_DTW, alternative = "greater")
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  FDTW and CID_DTW
## W = 3244, p-value = 0.8166
## alternative hypothesis: true location shift is greater than 0

Critical difference

Classification algorithme are sort based on their average accuracy. FDTW is rank 20th/37 algorithms considered

library(scmamp)

plotCD (d[,2:37], alpha=0.05, cex=0.7)