A10: Multilayer perceptrons
This assignment helps you practice designing and training multilayer perceptrons.
On a dataset of your choice (including datasets we have used previously), use Weka to train multiple multilayer perceptrons (MLPs) with different configurations, listed below. Use the exact same dataset in each case. Summarize the results in the way described below.
- different MLP network designs (“hiddenLayers” attribute in Weka MLP options); try no hidden layers, one hidden layer, two hidden layers, etc.
- epoch count (“trainingTime” in Weka)
- learning rate
Run at least 3 variations of each (so at least 9 experiments). Measure the accuracy (weighted avg. Precision, Recall, and F-Measure) and training time for each configuration. The training time can be seen in Weka just above the summary; it is written as: “Time taken to build model: XYZ seconds”.
Summarize the results by answering the following questions:
- What effect does network design have on training time and accuracy?
- What effect does epoch count have on training time and accuracy?
- What effect does learning rate have on training time and accuracy?
- What configuration of all three parameters is best? Make an argument that considers training time (cost) and accuracy (benefit); or argue that there is no clear winner.
Note: your discoveries may only apply to the dataset you’ve chosen, and do not generalize to all use cases for MLPs. Be aware that each dataset may need its own unique MLP configuration.
Submit each of the following:
- Your dataset or a link to it.
- A table (perhaps a spreadsheet) of all of your configurations plus the accuracy and training time of each.
- Your written summary of the results (see above for requirements).
You will receive full credit if you perform all of the experiments correctly and your summary makes sense given your results. Points will be taken off if you are missing any of these deliverables, your experiments are incomplete, and/or your summary does not make sense for your results.