ToolboxPerformance: Difference between revisions

From Eigenvector Research Documentation Wiki
Jump to navigation Jump to search
imported>Mathias
imported>Donal
No edit summary
 
(13 intermediate revisions by one other user not shown)
Line 1: Line 1:
==PLS_Toolbox Performance==
==PLS_Toolbox Performance==


The following performance results are for general comparison and expectation. Your own mileage may vary.
This page presents performance results from the main regression methods in PLS_Toolbox to allow comparison of their time and memory requirements when analyzing datasets of various sizes. These results were obtained using a Mac OSX (El Capitan) computer with a 2.8 GHz Intel CPU and 16 GB RAM. Matlab version R2015a and PLS_Toolbox version 8.1 were used.
 
 
{| class="wikitable"
|+ Performance Table
! Matlab Versoin !! PLS_Toolbox Version !! Operating System  !! System Description|| Data Description !! Algorithm !! Performance Result
|-
| 2015a || 8.1.1 || OS X El Capitan || 2.8 GHz Intel, 16 GB ram || cell || cell
|}


All results were obtained by building models from the command line using synthetic random datasets. Autoscale preprocessing was used with the default option values for all methods.  Cross-validation was not performed.   






 
'''PCA time in seconds required to build model'''
'''Table 1. PCA'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
Line 24: Line 16:
| |'''20000 samples'''
| |'''20000 samples'''
||  2
||  2
||  4.7  
||  4.7  
||  40  
||  40  


|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
||  3.5
||  3.5
||  9
||  9
||  61
||  61
|-
|-
|}
|}


''' PCA memory requirements'''
''' PCA memory requirements'''
Line 53: Line 35:
|- valign="top"  
|- valign="top"  
| |'''20000 samples'''
| |'''20000 samples'''
||  .2 GB
||  0.2 GB
 
||  1 GB
||  1 GB
||  3.5 GB
||  3.5 GB


|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
|| 3.55  
|| 3.55  
|| 9 GB
|| 9 GB
 
|| 10.5 GB
|| 61 GB
 
 


|-
|-
Line 75: Line 50:




 
'''PCR time in seconds required to build model'''
 
'''PCR'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
Line 86: Line 59:
| |'''20000 samples'''
| |'''20000 samples'''
||  3
||  3
||  6  
||  6  
||  44  
||  44  
|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
||  5
||  5
||  12
||  12
||  71
||  71
|-
|-
|}
|}


'''PCR memory requirements'''
'''PCR memory requirements'''
Line 112: Line 74:


| ||'''1000 variables'''||'''2000 variables'''||'''5000 variables'''
| ||'''1000 variables'''||'''2000 variables'''||'''5000 variables'''
|- valign="top"  
|- valign="top"  
| |'''20000 samples'''
| |'''20000 samples'''
|| .2 GB
|| 0.2 GB
 
||  1 GB
||  1  
||  4 GB  
 
||  2.8  
 
 
|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
||  .5 GB
||  0.5 GB
 
||  4 GB
||  3.8
 
||  11
||  11
|-
|}






|-
''' PLS time in seconds required to build model'''
|}
''' PLS'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
Line 143: Line 98:
| |'''20000 samples'''
| |'''20000 samples'''
|| 3.3
|| 3.3
|| 8   
|| 8   
|| 43   
|| 43   
|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
|| 8
|| 8
|| 18
|| 18
|| 98
|| 98
|-
|-
|}
|}


'''PLS Memory requirements'''
'''PLS Memory requirements'''
Line 168: Line 113:


| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
|- valign="top"  
|- valign="top"  
| |'''100 samples'''
| |'''100 samples'''
|| 1 GB
|| 1 GB
|| 2 GB
|| 2 GB
|| 5 GB  
|| 5 GB  
|- valign="top"  
|- valign="top"  
| |'''500 samples'''
| |'''500 samples'''
|| 1.6 GB
|| 1.6 GB
|| 5.2 GB
|| 5.2 GB
|| 13 GB
|| 13 GB
|-
|-
|}
|}
Line 194: Line 128:




 
'''LWR time in seconds required to build model'''
'''LWR'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
| ||'''1000 variables'''||'''5000 variables'''||'''10000 variables'''
| ||'''1000 variables'''||'''5000 variables'''||'''10000 variables'''
|- valign="top"  
|- valign="top"  
| |'''20000 samples'''
| |'''20000 samples'''
||  4
||  4
|| 65   
|| 65   
|| 76   
|| 76   
|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
|| 10
|| 10
|| 77
|| 77
||  670
||  670
|-
|-
|}
|}


''' LWR memory requirements'''
''' LWR memory requirements'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
| ||'''1000 variables'''||'''2000 variables'''||'''5000 variables'''
| ||'''1000 variables'''||'''2000 variables'''||'''5000 variables'''
|- valign="top"  
|- valign="top"  
| |'''20000 samples'''
| |'''20000 samples'''
||  <1 GB  
||  <1 GB  
||  2 GB  
||  2 GB  
 
||  3.5 GB  
||  3.4 GB  
 
 
|- valign="top"  
|- valign="top"  
| |'''50000 samples'''
| |'''50000 samples'''
||  .6 GB
||  0.6 GB
 
||  3 GB
||  3 GB
||  6.75 GB
||  6.75 GB
|-
|-
|}
|}
Line 256: Line 164:




 
'''ANN time in seconds required to build model'''
'''Table 3. ANN'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
|- valign="top"  
|- valign="top"  
| |'''500 samples'''
| |'''500 samples'''
||  6
||  6
|| 28   
|| 28   
|| 95   
|| 95   
|- valign="top"  
|- valign="top"  
| |'''1000 samples'''
| |'''1000 samples'''
|| 10
|| 10
||  370
||  370
||  360
||  360
|- valign="top"  
|- valign="top"  
| |'''2000 samples'''
| |'''2000 samples'''
||  12
||  12
||  550
||  550
 
||  2810
||  2810 s
 
 
|-
|-
|}
|}
Line 295: Line 188:




'''Table 4. SVM'''
'''SVM time in seconds required to train model'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
 
| ||'''100 variables'''||'''500 variables'''||'''2000 variables'''
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
 
|- valign="top"  
|- valign="top"  
| |'''100 samples'''
| |'''100 samples'''
||
|| 8
 
|| 28 
||  
|| 105 
 
||  
 
 
|- valign="top"  
|- valign="top"  
| |'''500 samples'''
| |'''500 samples'''
||
|| 150
 
|| 640
||
|| 2370
 
||
 
 
|- valign="top"  
|- valign="top"  
| |'''1000 samples'''
| |'''1000 samples'''
||
||
||
||
||
||
|-
|-
|}
|}




'''SVM with PCA compression'''
'''SVM with PCA compression time in seconds required to build model'''


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
|- valign="top"  
|- valign="top"  
| |'''100 samples'''
| |'''100 samples'''
||
|| 4
 
|| 4 
||  
|| 4
 
||  
 
 
|- valign="top"  
|- valign="top"  
| |'''500 samples'''
| |'''500 samples'''
||
|| 38
 
|| 38
||
|| 38
 
||
 
 
|- valign="top"  
|- valign="top"  
| |'''1000 samples'''
| |'''1000 samples'''
||
||
||
||
||
||
|-
|-
|}
|}
Line 371: Line 236:


{| border="1" cellpadding="5" cellspacing="0"
{| border="1" cellpadding="5" cellspacing="0"
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
| ||'''100 variables'''||'''500 variables'''||'''1000 variables'''
|- valign="top"  
|- valign="top"  
| |'''100 samples'''
| |'''100 samples'''
||
||
||   
||   
||   
||   
|- valign="top"  
|- valign="top"  
| |'''500 samples'''
| |'''500 samples'''
||
||
||
||
||
||
|- valign="top"  
|- valign="top"  
| |'''1000 samples'''
| |'''1000 samples'''
||
||
||
||
||
||
|-
|-
|}
|}

Latest revision as of 22:35, 13 September 2016

PLS_Toolbox Performance

This page presents performance results from the main regression methods in PLS_Toolbox to allow comparison of their time and memory requirements when analyzing datasets of various sizes. These results were obtained using a Mac OSX (El Capitan) computer with a 2.8 GHz Intel CPU and 16 GB RAM. Matlab version R2015a and PLS_Toolbox version 8.1 were used.

All results were obtained by building models from the command line using synthetic random datasets. Autoscale preprocessing was used with the default option values for all methods. Cross-validation was not performed.


PCA time in seconds required to build model

1000 variables 2000 variables 5000 variables
20000 samples 2 4.7 40
50000 samples 3.5 9 61

PCA memory requirements

1000 variables 2000 variables 5000 variables
20000 samples 0.2 GB 1 GB 3.5 GB
50000 samples 3.55 9 GB 10.5 GB


PCR time in seconds required to build model

1000 variables 2000 variables 5000 variables
20000 samples 3 6 44
50000 samples 5 12 71

PCR memory requirements

1000 variables 2000 variables 5000 variables
20000 samples 0.2 GB 1 GB 4 GB
50000 samples 0.5 GB 4 GB 11


PLS time in seconds required to build model

1000 variables 2000 variables 5000 variables
20000 samples 3.3 8 43
50000 samples 8 18 98

PLS Memory requirements

100 variables 500 variables 1000 variables
100 samples 1 GB 2 GB 5 GB
500 samples 1.6 GB 5.2 GB 13 GB


LWR time in seconds required to build model

1000 variables 5000 variables 10000 variables
20000 samples 4 65 76
50000 samples 10 77 670

LWR memory requirements

1000 variables 2000 variables 5000 variables
20000 samples <1 GB 2 GB 3.5 GB
50000 samples 0.6 GB 3 GB 6.75 GB


ANN time in seconds required to build model

100 variables 500 variables 1000 variables
500 samples 6 28 95
1000 samples 10 370 360
2000 samples 12 550 2810


SVM time in seconds required to train model

100 variables 500 variables 2000 variables
100 samples 8 28 105
500 samples 150 640 2370
1000 samples


SVM with PCA compression time in seconds required to build model

100 variables 500 variables 1000 variables
100 samples 4 4 4
500 samples 38 38 38
1000 samples

SVM memory requirements

100 variables 500 variables 1000 variables
100 samples
500 samples
1000 samples
Retrieved from "https://www.wiki.eigenvector.com/index.php?title=ToolboxPerformance&oldid=9577"