C++ comma operator and parallelization

I have never had a reason to use the comma operator, however, writing some modern code, it seems required. Say you have some series of variables and you want to perform a common operation on the group. New C++ is always a fun thing, I know. We could change this to be a generic functionContinue reading “C++ comma operator and parallelization”

Errors and Progress in mnist recognition

So, in my earlier post, I said I had a “convergent mnist network”. At the time I was excited and I wrote that in haste. What that network had been doing, it had been trained on null and digits, but only one image for each of these digit was actually ever trained into the network.Continue reading “Errors and Progress in mnist recognition”

Non-Convolutional Image Recognition

I have had some difficulty determining numbers for training times for mnist, so I am going to post some of mine, and also discuss what my network is doing. So far in my work on mnist, I have generated a convergent network. Using cpu only, ryzen 7 1700, I train a convergent network in underContinue reading “Non-Convolutional Image Recognition”

Code Simplicity of binary files and C++ wonder

There is a lot of code online about reading MNIST dataset, and I have produced my own version, which uses my Binary/Reader classes talked about in a previous post. So, you could be thinking, Binary Image file stuffs with C++, oh, no! I have seen some of the parsers easily obtainable online, written in C++,Continue reading “Code Simplicity of binary files and C++ wonder”

Movement to OpenCl++

In my project, I have been thinking about something, data-oriented-design, quite a lot during the creation of new code. I’m going to present some of these concepts in current implementation and then describe moving to a specifically DOD language, opencl++. Moving into OpenCl++ seems like a natural extention of DOD C++ I use. So, inContinue reading “Movement to OpenCl++”

Backpropagation is a class of random search algorithms

This is evident when watching networks learn using my visualisation program. The backpropagation algorithm creates spontaneous types of patterns which are put into or scuplted into the network briefely or for some period, and then there is a random shift to a new concept with a new pattern for its own breifity. This process repeatsContinue reading “Backpropagation is a class of random search algorithms”

Visualizing network training

Here is a video of where my project is at, Basically, it provides a graphical visualization of the network weights. I am going to experiment with generating and examining this sort of image through training. In this video, the network is training on six problems, and it slowly learns for about 45 seconds. After that,Continue reading “Visualizing network training”

Intelligent sequences of numbers

Since a large property of a neural network is indeed its initial or current random state, maybe we can consider this entire state as just a vector of randomly generated values. In reality, that’s what it is on the computer. Each of these values is generated via a random number generator in a specific sequence.Continue reading “Intelligent sequences of numbers”

Configurations of random variables

In my neural network program, I refactored some code and produced an error which I did not notice for some time. When I would run the program, eventually, out of 100 networks, a single or few networks would learn the pathing problem. With no difference in how they are trained, they are all taught exactlyContinue reading “Configurations of random variables”

Writing custom Random Number Generators and performance Comparisions

For some reason, I became concerned about the amount of time being spent by random number generators, so I took a look into them. I wrote a program which compares the speed of std::mt19937 ( mersenne twister ), a custom “twister loop”, a custom “pcg”, and custom “xorshift” generators. The standard twister was the slowest,Continue reading “Writing custom Random Number Generators and performance Comparisions”