Just a girl in Tech

Ok, so you read a bunch of stuff on how to do Neural Networks and how many layers or nodes you should add, and etc... But when you start to implement the actual Neural Networks you face a ton of dummy errors that stop your beautiful inspirational programming. This post talks about some errors you might face when using the neuralnet package in R. First, remember, to use the package you should install it: install.packages("neuralnet") Then library(" neuralnet") to load the package. Error 1 One error that might happen training your neural network is this: nn <- neuralnet(formula1,data=new_data, hidden=c(5,3)) Error in terms.formula(formula) : invalid model formula in ExtractVars This happens when the name of the variables in formula "formula1" are in a non desired format. For example if you named your columns (or variables) as numbers you would get this error. So change your column names and re-run the model! Example: label ~ 1 ...

When trying to interpolate a series of data the cubic spline  is a great technique to be used. I choose to use the smooth.spline function, from the R stats package. > smooth.spline(data$x,  data$y ) Nevertheless, while running smooth.spline on a collection of datasets with different sizes I got the following error: Error in smooth.spline( data$x,  data$y),  :   'tol' must be strictly positive and finite After digging a little bit I discovered that the problem was that some datasets were really small and smooth.spline wasn't being able to compute anything. Hence, make sure your dataset is big enough before applying smooth.spline to it. > if(length(data$x) > 30) {  smooth.spline( data$x,  data$y)  } UPDATE:  A more generalized solution would be: > if(IQR(data$x) > 0) {  smooth.spline( data$x,  data$y)  }

When dealing with a significant amount of data in R the are some points to consider. How do I know if my data is too big? Well, the term "BigData" can be thought of as a data that is too big to fit in the available memory. As R works with the entire dataset in memory (unless you specify it not to do so), the first thing is to check how large is the dataset in question, and if it does fit in memory . Remember that you actually should have at least double memory of the size of your dataset. So for example if you dataset has a size of 2 GB, you should have at least 4 GB of memory. If you don't have enough memory, you should consider breaking your data into smaller chunks and working with them separately. You can use the command split to do this in Linux: split -l 10000 file.txt new_file This should create several new files (new_filea, new_fileb, etc..) with ten thousand lines each. Well, once you know your date will fit into memory, you can read it with th...

I am currently working a lot with R . One simple thing that helps me to better visualize data is to plot it excluding outliers. To do so, first read the data data = read.table(“myfile.txt”)                                      Then, you can check how data is distributed quantile(data, c(.02, .05, .10, .50, .90, .95, .98))                 An example output would be    2%   5%  10%  50%  90%  95%  98%   189  190 190  194  241  275  316  Now,  to plot your data discarding the 1% lowest values and 1% higher values, you could use x <- quantile(data, c(.01, .99))                                    And then plot(data, xlim=c(x[[1]], x[[2]])) ...

Many Machine Learning articles and papers describe the wonders of the Support Vector Machine (SVM) algorithm. Nevertheless, when using it on real data trying to obtain a high accuracy classification, I stumbled upon several issues. I will try to describe the steps I took to make the algorithm work in practice. This model was implemented using R and the library "e1071". To install and use it type: > install.packages("e1071") > library("e1071") When you want to classify data in two categories , few algorithms are better than SVM. It usually divides data in two different sets by finding a "line" that better separates the points. It is capable to classify data linearly (put a straight line to differentiate sets) or do a nonlinear classification (separates sets with a curve). This "separator" is called a hyperplane . Picture 1 - Linear hyperplane separator Normalize Features Before you even start running the algor...