3y ago

Machine Learning Beginner Info/Resources

MOOCs

Nowadays, there are a couple of really excellent online lectures to get you started. The list is too long to include them all. Every one of the major MOOC sites offers not only one but several good Machine Learning classes, so please check coursera, edX, Udacity yourself to see which ones are interesting to you.

However, there are a few that stand out, either because they're very popular or are done by people who are famous for their work in ML. Roughly in order from easiest to hardest, those are:

Andrew Ng's ML-Class at coursera: Focused on application of techniques. Easy to understand, but mathematically very shallow. Good for beginners!https://www.coursera.org/course/ml
Hasti/Tibshirani's Statistical Learninghttps://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about
Yaser Abu-Mostafa's Learning From Data: Focuses a lot more on theory, but also doable for beginnershttps://work.caltech.edu/telecourse.html
Geoff Hinton's Neural Nets for Machine Learning: As the title says, this is almost exclusively about Neural Networks.https://www.coursera.org/course/neuralnets
Hugo Larochelle's Neural Net lectures: Again mostly on Neural Nets, with a focus on Deep Learninghttp://www.youtube.com/playlist?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH
Daphne Koller's Probabilistic Graphical Models Is a very challenging class, but has a lot of good material that few of the other MOOCs here will coverhttps://www.coursera.org/course/pgm

Books

The most often recommended textbooks on general Machine Learning are (in no particular order):

Hasti/Tibshirani/Friedman's Elements of Statistical Learning FREEhttp://statweb.stanford.edu/%7Etibs/ElemStatLearn/
Barber's Bayesian Reasoning and Machine Learning FREEhttp://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage
MacKay's Information Theory, Inference and Learning Algorithms FREEhttp://www.inference.phy.cam.ac.uk/itila/book.html
Goodfellow/Bengio/Courville's Deep Learning FREEhttp://www.deeplearningbook.org/
Nielsen's Neural Networks and Deep Learning FREEhttp://neuralnetworksanddeeplearning.com/
Graves' Supervised Sequence Labelling with Recurrent Neural Networks FREEhttp://www.cs.toronto.edu/%7Egraves/preprint.pdf
Sutton/Barto's Reinforcement Learning: An Introduction; 2nd Edition FREEhttps://www.dropbox.com/s/7jl597kllvtm50r/book2015april.pdf

Note that these books delve deep into math, and might be a bit heavy for complete beginners. If you don't care so much about derivations or how exactly the methods work but would rather just apply them, then the following are good practical intros:

An Introduction to Statistical Learning FREEhttp://www-bcf.usc.edu/%7Egareth/ISL/
Probabilistic Programming and Bayesian Methods for Hackers FREEhttp://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Prologue/Prologue.ipynb

There are of course a whole plethora on books that only cover specific subjects, as well as many books about surrounding fields in Math. A very good list has been collected by /u/ilsunil here

Deep Learning Resources

Karpathy's Stanford CS231n: Convolutional Neural Networks for Visual Recognition (Lecture Notes)http://cs231n.github.io/
Video Lecture's Stanford CS231n: Convolutional Neural Networks for Visual Recognitionhttps://www.youtube.com/playlist?list=PLlJy-eBtNFt6EuMxFYRiNRS07MCWN5UIA
Silver's Reinforcement Learning Lectureshttps://www.youtube.com/watch?v=2pWv7GOvuf0
Colah's Informational Bloghttp://colah.github.io/
Bruna's UC Berkeley Stat212b: Topics Course on Deep Learninghttps://joanbruna.github.io/stat212b/
Overview of Neural Network Architectureshttp://www.asimovinstitute.org/neural-network-zoo/

Math Resources

Strang's Linear Algebra Lectureshttps://www.youtube.com/watch?v=ZK3O402wf1c
Kolter/Do's Linear Algebra Review and Reference Noteshttp://cs229.stanford.edu/section/cs229-linalg.pdf
Calculus 1https://www.edx.org/course/calculus-1a-differentiation-mitx-18-01-1x
Introduction to Probabilityhttps://www.edx.org/course/introduction-probability-science-mitx-6-041x-1

Programming Languages and Software

In general, the most used languages in ML are probably Python, R and Matlab (with the latter losing more and more ground to the former two). Which one suits you better depends wholy on your personal taste. For R, a lot of functionality is either already in the standard library or can be found through various packages in CRAN. For Python, NumPy/SciPy are a must. From there, Scikit-Learn covers a broad range of ML methods.

If you just want to play around a bit and don't do much programming yourself then things like Visions of Chaos, WEKA, KNIME or RapidMiner might be of your liking. Word of caution: a lot of people in this subreddit are very critical of WEKA, so even though it's listed here, it is probably not a good tool to do anything more than just playing around a bit. A more detailed discussion can be found here

Deep Learning Software, GPU's and Examples

There are a number of modern deep learning toolkits you can utilize to implement your models. Below, you will find some of the more popular toolkits. This is by no means an exhaustive list. Generally speaking, you should utilize whatever GPU has the most memory, highest clock speed, and most CUDA cores available to you. This was the NVIDIA Titan X from the previous generation. These frameworks are all very close in computation speed, so you should choose the one you prefer in terms of syntax.

Theano is a python based deep learning toolkit developed by the Montreal Institute of Learning Algorithms, a cutting edge deep learning academic research center and home of many users of this forum. This has a large number of tutorials ranging from beginner to cutting edge research.

Torch is a Luajit based scientific computing framework developed by Facebook Artificial Intelligence Research (FAIR) and is also in use at Twitter Cortex. There is the torch blog which contains examples of the torch framework in action.

TensorFlow is a python deep learning framework developed by Google Brain and in use at Google Brain and Deepmind. The newest framework around. Some TensorFlow examples may be found here Do not ask questions on the Google Groups, ask them on stackoverflow

Neon is a python based deep learning framework built around a custom and highly performant CUDA compiler Maxas by NervanaSys.

Caffe is an easy to use, beginner friendly deep learning framework. It provides many pretrained models and is built around a protobuf format of implementing neural networks.

Keras can be used to wrap Theano or TensorFlow for ease of use.

Datasets and Challenges for Beginners

There are a lot of good datasets here to try out your new Machine Learning skills.

Kaggle has a lot of challenges to sink your teeth into. Some even offer prize money!http://www.kaggle.com/
The UCI Machine Learning Repository is a collection of a lot of good datasetshttp://archive.ics.uci.edu/ml/
http://blog.mortardata.com/post/67652898761/6-dataset-lists-curated-by-data-scientists lists some more datasets
Here is a very extensive list of large-scale datasets of all kinds.http://www.quora.com/Data/Where-can-I-find-large-datasets-open-to-the-public
Another dataset listhttp://www.datawrangling.com/some-datasets-available-on-the-web

Research Oriented Datasets

In many papers, you will find a few datasets are the most common. Below, you can find the links to some of them.

MNIST A short handwriting dataset that is often used as a sanity check in modern researchhttp://yann.lecun.com/exdb/mnist/
SVHN Similar to MNIST, but with color numbers. A sanity check in most cases.http://ufldl.stanford.edu/housenumbers/
CIFAR-10/0 CIFAR 10 and 100 are two natural color images that are often used with convolutional neural networks for image classification.https://www.cs.toronto.edu/%7Ekriz/cifar.html

Communities

http://www.datatau.com/ is a data-science centric hackernews
http://metaoptimize.com/qa/ and http://stats.stackexchange.com/ are Stackoverflow-like discussion forums

ML Research

Machine Learning is a very active field of research. The two most prominent conferences are without a doubt NIPS and ICML. Both sites contain the pdf-version of the papers accepted there, they're a great way to catch up on the most up-to-date research in the field. Other very good conferences include UAI (general AI), COLT (covers theoretical aspects) and AISTATS.

Good journals for ML papers are the Journal of Machine Learning Research, the Journal of Machine Learning and arxiv.

FAQ

How much Math/Stats should I know?

That depends on how deep you want to go. For a first exposure (e.g. Ng's Coursera class) you won't need much math, but in order to understand how the methods really work,having at least an undergrad level of Statistics, Linear Algebra and Optimization won't hurt.