Read about current machine learning papers: https://paperswithcode.com/
\nLast layer is usually called L.
MAX_POOLING) - no parameters to learnConvolutional layers: Applying a convolution function with weights to learn and update
\nThe rectifier is a popular activation function - the positive part of its argument.
\n\n\nA unit employing the rectifier is also called a rectified linear unit (ReLU)
\n
In pytorch: res.max(tensor(0.0)) or F.relu (plot it: plot_function(F.relu))
Why ReLU? - Vanishing gradient problem. Easy to compute.
\nLeaky ReLU - instead of 0 for x < 0 -> choose slightly decreasing value
\nThe universal approximation theorem shows that any function can be approximated as closely as needed using just one nonlinearity. So why do we normally use more?
\nData Science Mindset
\nPrefer ugly pipeline over nicely implemented pipeline
\nEngineering Mindset
\nBuild nice pipeline
\nAccuracy/Precision: Correct vs. total amount of test cases (only works reliably for data sets where classes are well balanced).
\nRecall: Ability of a model to find all the relevant cases within a dataset:
\nF_1 score: Harmonic mean of recall and precision
\nCompare everything in a confusion matrix.
\nSpecificity: True Negative Rate
\n\n\n“Specificity” is often remembered as “SpPIn” (like “specifically pinning down” a disease or condition), as it refers to the proportion of true negative cases that are correctly identified by a test.
\n
\n\n“Specificity” - “SpPIn” (like “Specificity Pins Down” the true negatives)
\n
Sensitivity: True Positive Rate
\nReduce false negatives for a higher sensitivity.
\n\n\n“Sensitivity” is often remembered as “SnNout” (like “sniffing out” a disease or condition), as it refers to the proportion of true positive cases that are correctly identified by a test.]
\n
\n\n“Sensitivity” - “SeSeize” (like “Sensitivity Seizes” the true positives)
\n
\n\nHow to decide whether to prefer specificity or sensitivity?
\n
ROC curve: Plot sensitivity vs. 1 - specificity
\nAUC: Area under the ROC curve
\n- **Note**: AUC = 1: Perfect classifier\n- **Note**: AUC = 0.5: Random classifierAn iterator which collects the data and makes it ready to be trained by a neural network.
\nShow what weights are doing.. what does one of the first layers do/emphasize?
\nshow_image(w[2].view(28,28))Exclusive vs. non-exclusive classes.
\nUse one-hot-encoding to encode output layer.
\nHow to filter the output value of a neuron before passing it on.
\nNon-exclusive classes
\nUse Sigmoid activation function
Exclusive classes
\nSoftmax activation function to ensure that sum of all classes equals 1.Metric vs. Loss
\ne.g. …
\nCost/Loss functions
\nOptimized learning rate
\nw?w.dC/dw = dC/da * da/dz * dz/dw.dC/db to also compute how much a change in the bias impacts the cost.Stochastic gradient descent: Uses random mini-batches instead of calculating the gradient of the entire dataset.
\nA nice machine learning graphic:
\nWinning Kaggle Contributions are usually an ensemble of neural networks
\nDeep Learning with Attention.
\n0.45 and 0.55 percent of the height is defined.The image data comes in as a long array (height x width) and it’s overall (sampled at every third pixel) grayscale values (Hex values between #000 and #fff) are averaged.
Android: Inside a for loop over all pixels these grayScaleValues are summed up:
\ngrayScaleValue = data[i] & 0xFF;0.025 percent) at the top of the image is also recorded and used to normalize the values of the center (e.g. for cases in which the phone moves and thus the background changes)innerZeroCrossings - outerZeroCrossings. In fact the object which traverses the center (innerZeroCrossings) has to be more prominent than the grayscale changes in the background (outerZeroCrossings). Otherwise, the center (inner) grayscale change is just a copy of the grayscale changes in the periphery of the image.Adding non-linearity -> You can approximate any function if weights and bias are big enough.
\n