Difference between revisions of "MSChallenge"

From IACL
Jump to navigation Jump to search
(Update.)
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
<meta name="title" content="The 2015 Longitudinal Multiple Sclerosis Lesion Segmentation Challenge" />  
+
<!-- <meta name="title" content="The 2015 Longitudinal Multiple Sclerosis Lesion Segmentation Challenge" /> -->
 
{{h2|The 2015 Longitudinal MS Lesion Segmentation Challenge}}
 
{{h2|The 2015 Longitudinal MS Lesion Segmentation Challenge}}
 
{{MSChallengeNav}}
 
{{MSChallengeNav}}
Line 12: Line 12:
  
  
{{h3|Current Leaderboard}}
+
{{h3|Leaderboard & Paper}}
A live leaderboard is maintained on the [https://smart-stats-tools.org/node/26 Smart Stats Website]. This leaderboard is updated to include links to the associated papers; most currently point to the main Challenge Article: {{pub|author=A. Carass, S. Roy, A. Jog, J.L. Cuzzocreo, E. Magrath, A. Gherman, J. Button, J. Nguyen, F. Prados, C.H. Sudre, M.J. Cardoso, N. Cawley, O. Ciccarelli, C.A.M. Wheeler-Kingshott, S. Ourselin, L. Catanese, H. Deshpande, P. Maurel, O. Commowick, C. Barillot, X. Tomas-Fernandez, S.K. Warfield, S. Vaidya, A. Chunduru, R. Muthuganapathy, G. Krishnamurthi, A. Jesson, T. Arbel, O. Maier, H. Handels, L.O. Iheme, D. Unay, S. Jain, D.M. Sima, D. Smeets, M. Ghafoorian, B. Platel, A. Birenbaum, H. Greenspan, P.-L. Bazin, P.A. Calabresi, C.M. Crainiceanu, L.M. Ellingsen, D.S. Reich, J.L. Prince, and D.L. Pham|title=Longitudinal Multiple Sclerosis Lesion Segmentation: Resource and Challenge|jrnl=ni|number=148(C):77-102|when=2017|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490}}.
+
'''A live leaderboard is maintained on the [https://smart-stats-tools.org/lesion-challenge Smart Stats Website].''' The main Challenge article has appeared in {{iacl-pub ni}}: {{iacl-pub|author=A. Carass, S. Roy, A. Jog, J.L. Cuzzocreo, E. Magrath, A. Gherman, J. Button, J. Nguyen, F. Prados, C.H. Sudre, M.J. Cardoso, N. Cawley, O. Ciccarelli, C.A.M. Wheeler-Kingshott, S. Ourselin, L. Catanese, H. Deshpande, P. Maurel, O. Commowick, C. Barillot, X. Tomas-Fernandez, S.K. Warfield, S. Vaidya, A. Chunduru, R. Muthuganapathy, G. Krishnamurthi, A. Jesson, T. Arbel, O. Maier, H. Handels, L.O. Iheme, D. Unay, S. Jain, D.M. Sima, D. Smeets, M. Ghafoorian, B. Platel, A. Birenbaum, H. Greenspan, P.-L. Bazin, P.A. Calabresi, C.M. Crainiceanu, L.M. Ellingsen, D.S. Reich, J.L. Prince, and D.L. Pham|title=Longitudinal Multiple Sclerosis Lesion Segmentation: Resource and Challenge|jrnl=ni|number=148(C):77-102|when=2017|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490}}<br />
 +
and a companion paper has appeared in ''Data in Brief'':
 +
{{iacl-pub|author=A. Carass, S. Roy, A. Jog, J.L. Cuzzocreo, E. Magrath, A. Gherman, J. Button, J. Nguyen, P.-L. Bazin, P.A. Calabresi, C.M. Crainiceanu, L.M. Ellingsen, D.S. Reich, J.L. Prince, and D.L. Pham|title=Longitudinal multiple sclerosis lesion segmentation data resource|journal=Data in Brief|number=12:346-350|when=2017|doi=10.1016/j.dib.2017.04.004|pubmed=28491937}}
 +
 
 +
 
 +
<!--
 
{| align="center" style="width:80%; border:2px #87cefa solid; background:#f0ffff; text-align:left;"
 
{| align="center" style="width:80%; border:2px #87cefa solid; background:#f0ffff; text-align:left;"
 
|-
 
|-
Line 33: Line 38:
 
|
 
|
 
| 91.267
 
| 91.267
 +
|-
 +
| colspan="4" style="border-top:1px solid #87cefa;"|
 +
|-
 +
| 2
 +
| ''MIAC''<br />S. Andermatt, J. Würfel, & P.C. Cattin<br />''' '''<br />
 +
|
 +
| 91.011
 
|-
 
|-
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
Line 51: Line 63:
 
|-
 
|-
 
| 5
 
| 5
| ''ATMS''<br />Olfa Ghribi<br /><br />
+
| ''ATMS''<br />O. Ghribi<br /><br />
 
|
 
|
 
| 90.170
 
| 90.170
Line 65: Line 77:
 
|-
 
|-
 
| 7
 
| 7
| ''ATMS''<br />Olfa Ghribi<br /><br />
 
|
 
| 89.844
 
|-
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
| 8
 
 
| ''Team VISAGES GCEM''<br />L. Catanese, O. Commowick, & C. Barillot<br />'''Automatic Graph Cut Segmentation of Multiple Sclerosis Lesions'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team VISAGES GCEM''<br />L. Catanese, O. Commowick, & C. Barillot<br />'''Automatic Graph Cut Segmentation of Multiple Sclerosis Lesions'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 78: Line 83:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 9
+
| 8
 
| ''Team IIT Madras''<br />S. Vaidya, A. Chunduru, R. Muthuganapathy, & G. Krishnamurthi<br />'''Longitudinal Multiple Sclerosis Lesion Segmentation using 3D Convolutional Neural Networks'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team IIT Madras''<br />S. Vaidya, A. Chunduru, R. Muthuganapathy, & G. Krishnamurthi<br />'''Longitudinal Multiple Sclerosis Lesion Segmentation using 3D Convolutional Neural Networks'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 85: Line 90:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 10
+
| 9
 
| ''Team MS*metrix*''<br />S. Jain, D.M. Sima, & D. Smeets<br />'''Automatic Longitudinal Multiple Sclerosis Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team MS*metrix*''<br />S. Jain, D.M. Sima, & D. Smeets<br />'''Automatic Longitudinal Multiple Sclerosis Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 92: Line 97:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 11
+
| 10
 
| ''Lesion-TOADS''<br />N. Shiee, P.-L. Bazin, A. Ozturk, D. S. Reich, P. A. Calabresi, & D. L. Pham<br />'''A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions'''<br />{{pub|doi=10.1016/j.neuroimage.2009.09.005|pubmed=19766196|pmcid=2806481|period=}}
 
| ''Lesion-TOADS''<br />N. Shiee, P.-L. Bazin, A. Ozturk, D. S. Reich, P. A. Calabresi, & D. L. Pham<br />'''A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions'''<br />{{pub|doi=10.1016/j.neuroimage.2009.09.005|pubmed=19766196|pmcid=2806481|period=}}
 
|
 
|
Line 99: Line 104:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 12
+
| 11
 
| ''Team CMIC''<br />F. Prados, M.J. Cardoso, N. Cawley, O. Ciccarelli, C.A.M. Wheeler-Kingshott, & S. Ourselin<br />'''Multi-Contrast PatchMatch Algorithm for Multiple Sclerosis Lesion Detection'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team CMIC''<br />F. Prados, M.J. Cardoso, N. Cawley, O. Ciccarelli, C.A.M. Wheeler-Kingshott, & S. Ourselin<br />'''Multi-Contrast PatchMatch Algorithm for Multiple Sclerosis Lesion Detection'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 106: Line 111:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 13
+
| 12
 
| ''MORF''<br />A. Jog, A. Carass, D.L. Pham, & J.L. Prince<br />'''Multi-Output Random Forests for Lesion Segmentation in Multiple Sclerosis'''<br />{{pub|doi=10.1117/12.2082157|pubmed=27695155|pmcid=5041594|period=}}
 
| ''MORF''<br />A. Jog, A. Carass, D.L. Pham, & J.L. Prince<br />'''Multi-Output Random Forests for Lesion Segmentation in Multiple Sclerosis'''<br />{{pub|doi=10.1117/12.2082157|pubmed=27695155|pmcid=5041594|period=}}
 
|
 
|
Line 113: Line 118:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 14
+
| 13
 
| ''Team TIG-BF''<br />C.H. Sudre, M.J. Cardoso, & S. Ourselin<br />'''Model Selection Propagation for Application on Longitudinal MS Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team TIG-BF''<br />C.H. Sudre, M.J. Cardoso, & S. Ourselin<br />'''Model Selection Propagation for Application on Longitudinal MS Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 120: Line 125:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 15
+
| 14
 
| ''Team CRL''<br />X. Tomas-Fernandez & S.K. Warfield<br />'''Model of Population and Subject (MOPS) Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team CRL''<br />X. Tomas-Fernandez & S.K. Warfield<br />'''Model of Population and Subject (MOPS) Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 127: Line 132:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 16
+
| 15
 
| ''Team DIAG''<br />M. Ghafoorian & B. Platel<br />'''Convolution Neural Networks for MS Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team DIAG''<br />M. Ghafoorian & B. Platel<br />'''Convolution Neural Networks for MS Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 134: Line 139:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 17
+
| 16
 
| ''Team TIG''<br />C.H. Sudre, M.J. Cardoso, & S. Ourselin<br />'''Model Selection Propagation for Application on Longitudinal MS Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team TIG''<br />C.H. Sudre, M.J. Cardoso, & S. Ourselin<br />'''Model Selection Propagation for Application on Longitudinal MS Lesion Segmentation'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 141: Line 146:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 18
+
| 17
 
| ''Team VISAGES DL''<br />H. Deshpande, P. Maurel, & C. Barillot<br />'''Sparse Representations and Dictionary Learning Based Longitudinal Segmentation of Multiple Sclerosis Lesions'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team VISAGES DL''<br />H. Deshpande, P. Maurel, & C. Barillot<br />'''Sparse Representations and Dictionary Learning Based Longitudinal Segmentation of Multiple Sclerosis Lesions'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 148: Line 153:
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
| colspan="4" style="border-top:1px solid #87cefa;"|
 
|-
 
|-
| 19
+
| 18
 
| ''Team BAUMIP''<br />L.O. Iheme & D. Unay<br />'''Automatic White Matter Hyperintensity Segmentation using FLAIR MRI'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
| ''Team BAUMIP''<br />L.O. Iheme & D. Unay<br />'''Automatic White Matter Hyperintensity Segmentation using FLAIR MRI'''<br />{{pub|doi=10.1016/j.neuroimage.2016.12.064|pubmed=28087490|period=}}
 
|
 
|
Line 156: Line 161:
 
|-
 
|-
 
|}
 
|}
 +
-->
  
  

Latest revision as of 00:59, 3 July 2022

The 2015 Longitudinal MS Lesion Segmentation Challenge

2015 Longitudinal MS Lesion Segmentation Challenge
MS Challenge Overview MS Challenge Data MS Challenge Evaluation

I. Introduction

2015 Longitudinal Multiple Sclerosis Lesion Segmentation Challenge

The Longitudinal MS Lesion Segmentation Challenge was conducted at the 2015 International Symposium on Biomedical Imaging in New York, NY, April 16-19. Competing teams applied their automatic lesion segmentation algorithms to MR neuroimaging data acquired at multiple time points from MS patients. Algorithms were evaluated against manual segmentations from two raters in terms of their segmentation accuracy and ability to track lesion evolution.

34 Teams initially registered for the Challenge coming from 15 different countries, representing 27 different institutions/universities. Congratulations to Team IIT Madras (First Prize), Team PVG_1 (Second Prize), and Team IMI (Third Prize and Efficiency Prize)!

Information about the data is available here, and the evaluation software from here.


Leaderboard & Paper

A live leaderboard is maintained on the Smart Stats Website. The main Challenge article has appeared in NeuroImage:

  • A. Carass, S. Roy, A. Jog, J.L. Cuzzocreo, E. Magrath, A. Gherman, J. Button, J. Nguyen, F. Prados, C.H. Sudre, M.J. Cardoso, N. Cawley, O. Ciccarelli, C.A.M. Wheeler-Kingshott, S. Ourselin, L. Catanese, H. Deshpande, P. Maurel, O. Commowick, C. Barillot, X. Tomas-Fernandez, S.K. Warfield, S. Vaidya, A. Chunduru, R. Muthuganapathy, G. Krishnamurthi, A. Jesson, T. Arbel, O. Maier, H. Handels, L.O. Iheme, D. Unay, S. Jain, D.M. Sima, D. Smeets, M. Ghafoorian, B. Platel, A. Birenbaum, H. Greenspan, P.-L. Bazin, P.A. Calabresi, C.M. Crainiceanu, L.M. Ellingsen, D.S. Reich, J.L. Prince, and D.L. Pham, "Longitudinal Multiple Sclerosis Lesion Segmentation: Resource and Challenge", NeuroImage, 148(C):77-102, 2017. (doi) (PubMed)

and a companion paper has appeared in Data in Brief:

  • A. Carass, S. Roy, A. Jog, J.L. Cuzzocreo, E. Magrath, A. Gherman, J. Button, J. Nguyen, P.-L. Bazin, P.A. Calabresi, C.M. Crainiceanu, L.M. Ellingsen, D.S. Reich, J.L. Prince, and D.L. Pham, "Longitudinal multiple sclerosis lesion segmentation data resource", Data in Brief, 12:346-350, 2017. (doi) (PubMed)



VII. Organizers

Primary Organizer:
Dzung Pham, Center for Neuroscience and Regenerative Medicine, Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD

Organizing committee members:
Pierre-Louis Bazin, Department of Neurophysics, Max Planck Institute, Leipzig, Germany
Aaron Carass, Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD
Peter Calabresi, Department of Neurology, Johns Hopkins University, Baltimore, MD
Ciprian Crainiceanu, Department of Biostatistics, Johns Hopkins University, Baltimore, MD
Lotta Ellingsen, Department of Electrical and Computer Engineering, University of Iceland, Reykjavik, Iceland
Qing He, Center for Neuroscience and Regenerative Medicine, Henry Jackson Foundation, Bethesda, MD
Jerry Prince, Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD
Daniel Reich, Translational Neuroradiology Unit, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD
Snehashis Roy, Center for Neuroscience and Regenerative Medicine, Henry Jackson Foundation, Bethesda, MD


VIII. Funding support

This challenge was supported in part by a grant from the National Institute of Neurological Disorders and Stroke (NINDS R01 NS070906). Funding for prizes was provided by the National Multiple Sclerosis Society.