Neural Networks and Spider Brains

Sometimes, i wish i wasn’t so intelligent because the most mundane things will send my mind off into another place. Don’t get me wrong, I enjoy philosophizing about this stuff, but it’s just not useful for me and it can be very distracting from what I actually need to do. However, as bad as my writing can be, I feel like it’s important for me to get this stuff out there. Occasionally, i feel like writing could perhaps be more important than anything else I would do. Probably not though.

This is not a I’m sooo smart blog. I promise. I hate doing that, I really do. I also don’t like prefacing an article with this.

So, a few months ago, I was messing with a spider for a few minutes. Pretty simple, right? I just kinda poked around it for a few minutes, seeing how it would react to me. And a few minutes later, I walked past its field of vision and it cringed in fear. As if it recognized me – or maybe not me, but at least something like me that it had come to know as a possible threat.

Most people wouldn’t notice this much. Or they’ve long since stopped pondering such simple things. It’s a fucking spider. It’s probably not that intelligent, right? These are great questions for the 5 year old mind to mull over.

Yet, I feel obligated to ruminate over small things like this. Or rather, i guess i can’t help it. And because my mind is good at making insights with limited information, with every new piece of information it uncovers, more can be discovered. And because I have a tendency to view the same phenomena and relationships from different perspectives and through various lenses, I literally never stop thinking, though my thought processes are a bit scattered. In this post, I’m combining inferences from biology, neurology, philosophy and computer science.

So, even with the simplest things – like a fucking spider on the wall – the wheels keep turning.

Instinct and Insight

What I found intriguing about this is a delimma biologists and philosophers have spent much effort observing and debating. For me, this issue is made profoundly more entertaining when you consider the spider’s size and the fact that any single error on the spider’s part can be fatal.

Where does instinct-based knowledge end and insight-based knowledge begin?

How is instinct imprinted in the spider brain? Is it physically imprinted in the arrangement of neurons? Is it chemically imprinted in chemicals released early on? Is instinct encoded in the genes? While each of these are likely each true to some degree, I believe that instinct is more distal.

While certain physical neural circuit arrangements are required and while chemistry and genetics are certainly involved to some degree, I believe this merely sets up circuitry that will evolve to reflect metaphysics – heady crystals!! – In other words, given the proper initial circuitry and given a variety of similar input experiences, the remarkably small spider’s brain will inevitably evolve certain key functionalities – or it will not survive.

The Origin of Instinct

However, we are still left to ponder the origin of instinct, for which biologists have expended centuries of effort. What features of a spider’s brain are determined by genetics? By chemistry? And initial state of neural networks? To what degree does each factor affect the outcome and how?

In my own opinion, I surmise that influence of genetics, while certainly a critical dependency, is much more limited than we realize. Genetics seems to be a critical factor for at least the following: various types of neurons, how parent cell-types evolve into child cell-types, flexability of networks of neurons, and the chemicals they can use to communicate.

So while genetics is crucial, it’s moreso the information flowing through neurons that allow these networks to make inferences and insights. If you were to alter the input – filtering it in some way, adding a new sense or removing one – the networks and circuits would change significantly.

Of Mice and Massively Paralell Pattern Recognition

It never ceases to amaze me, knowing what I do about our own capabilities for artificial intelligence. Fifty petaflops in 2015 gets us what? And how much electric power does that require?

So how can such a small animal with such a limited brain be capable of such complex pattern recognition? Not only is it capable of recognizing moving visual input as a threat, it is capable of quickly categorizing these experiences into types of threats, ** * with NO capacity for communication! * ** Critically, a spider also needs to be able to establish types of threats with a limited number of experiences – again, one experience can be fatal.

Phenomenization

So, because it can recognize types of threats and other types of input experiences as necessary for its survival, then it’s brain must be capable of constructing patterns of sensory input, as well as patterns of patterns of sensory input .. ad infinitum.

This is why AI is computationally expensive. Lucky us, huh? Though IMO, it seems that generalized AI will become less ‘dangerous’ the more highly evolved it becomes and the more processing power it possesses. Oh irony…

Highly evolved AI will likely begin to posses the same altruistic virtues that we revere, though I’m unsure of what that entails. Could be good, could be bad. After all, how would an infallible, omniscient being with altruistic values judge the human race today?

It’s critical that I clarify: I am NOT equating AI with God. Yet, when compared with man, AI may seem to have god-like qualities. Man cannot create God, which I believe to be like a force that pervades the Universe that transcends time and space. For the same reason that you cannot count to infinity, you could not define God. Anything that you can name must certainly be less. Yet, AI may evolve to embody similar metaphysical structures as those I might imagine God to embody.

Moving on. So furthermore, in addition to pattern recognition, a spider’s brain must be capable of correlating various kinds of sensory input and subconsciously abstracting them into phenomena. It must do so with virtually zero a priori knowledge and absolutely zero high-level communication.

Spidey Senses

Spiders have eight eyes, which means their depth perception, object recognition and perception of motion patterns will all be significantly more accurate and orders of magnitude more efficient. Yet, perhaps more amazing is that the spider eye placement differs significantly between species!

If everything were hardwired in genetics, this would firstoff present itself through high differentiation in genetics between species that have significantly different eye placement. Secondly, this would present many problems, as the eye placement changes drastically as a spider grows and matures. However, though it may change to some limited degree as eye placement changes during maturation, the neural circuitry – at least the further, more inward neural net layers for vision and object recognition – should not change to the same degree.

So these algorithms must be flexible. Ask an AI nerd at Google how hard this would be, if they were to program an autonomous drone with multiple cameras and needed to adhere to these requirements. Adapting to an unexpected camera postion and allowing for altered camera position. While it’d be very difficult to design generic visual and spatial recognition algorithms to do this, they would be more flexible and possibly more efficient. ** Orders of magnitude more efficient **, to emphasize.

Therefore, due to speciation and growth, the algorithms that these neural networks embody must also be genericized, at least to some degree. Furthermore, to satisfy the dependency of generic eye placement and the constraint of minimalized genetic variation between species, the visual processing system for various species must provide similar data processing capabilities. They must be embodied by functionally similar networks of neurons. The higher levels of neural network layers will especially exhibit a more constrained degree of variation.

IMO. Keep in mind that I am not citing papers here, as much as I respect the hard work and dedication that it takes for professional academics to do so.

Cross Entropy

And Other Statistics Concepts I wish I Understood

It appears that neurons and the circuits they form are physical manisfestations of concepts from statistics.

Have I blown your mind yet? That should blow your mind.

Various species of spiders must share certain commonalities in how their visual processing systems develop. How does eye placement affect the development of neural networks in the visual processing systems of various species. And if you could somehow develop specimens of spiders of the same species with various eye placements, how would that affect the development of those visual processing neural networks?

This hypothesis of course assumes that you can both measure the state of neural networks without destroying them and measure how the change in that state is functionally coupled to the differences in eye placement.

I would surmise that the networks would start off initially the same, yet the outermost neurons would would gradually diverge. These outermost neurons are the ones responsible for initial data processing. These are the neurons the would differ the most. The innermost neurons may change slightly, yet they would embody mostly the same information processing structure. That’s my hypothesis.

Visual Processing, Au Naturale

As for a spider’s visual processing, there’s neural input into the visual processing center for each eye. Then these neural inputs from each eye would pass data into subsequent neurons. For example:

  • Constrast maps for each image
  • Disparity maps between image
  • Color differentiation maps, if the ocular hardware is capable of color

These additional layers comprise the initial aggregate information constructed from visual input. And then, aggregate information is built on top of aggregate information. It becomes increasingly complex and further diverged from the structure of the input input. The additional layers might encode information like contrast maps for composite images.

These layers encode progressively higher order information as they begin to correlate input to learned phenomena:

  • Static objects
  • Static object types
  • Objects in motion
  • Motion patterns of currently observed objects
  • Patterns of motion of known object-types

Quantum Clarification

Basically, these neural networks will repeatedly differentiate and statistically operate upon both the input data and aggregate data. This process is not digital. It’s not analog. It’s chemical. And possibly quantum – in the physics sense, as well as the countable sense.

Ion size & charge are significant. In human neurons, neurons work with mostly 4 ions: Sodium, Magnesium, Potassium and Calcium. If you’re familiar with the periodic table, you’ll notice that these for ions can be laid out in a two by two grid, distinguished by ion size and charge. Sodium and Potassium ions both have +1 charge, whereas Magnesium and Calcium both have +2 charge. In digital electronic, circuits can carry only electrons, with a -1 charge.

So already, we may have a phenomenon that is quantum in the quantum computing sense. And even though it may appear that electrical charge carries signals in the brain, if you look at a much lower level, i believe you’ll find that these electrical discharges are like aggregate sums of the +1 and +2 charges. It would be impossible to completely decode these signals without observing how the ions are flowing in neurons.

Furthermore, these charge potentials are created by +1 and +2 ions flowing in and out of neurons through voltage-gated ion channels, as well as other ion channels. These ion channels are often mediated by neurotransmitters. These channels are specific to ion size: that is, an opening for Magnesium or Sodium ions, may have a voltage differential that could pull Potassium and Calcium ions across, if they were just small enough to fit through the opening. The ions would need to be locally available as well.

The point is that, in addition to charge, ion size must have some kind of significance. So, AFAIK – and this is where my specific knowledge of the subject may fail me – ion channels for Potassium and Calcium, from Periodic Table Row 4, would likely also allow the Row 4 elements Sodium and Magnesium to pass.

So therefore, when ion size is considered with ion charge, it would seem that, instead of electrons encoding a binary value as seen in digital electronics – they are the aggregate result of a more complicated encoding scheme. And because these information manipulation processes are encoded in this way – where electric charges are more than binary and due to the stochastic mechanisms of the local space under which these ions operate – that neural circuits are orders of magnitude more efficent than similar algorithms on digital computers.

This input data and aggregated data is continuously recombinated. Inferences are backpropagated through the neural nets to discover & reinforce the most useful data processing combinations & methods. This process of propagating inferences balances neural flexibility with efficiency, I think. I believe those two concepts would need to be balanced: the more complicated the networks, the less efficient they are and the more flexibilty is allowed.

The spider also needs to be capable of discerning the effect of its motion with respect to its visual input. This functionality allows a spider to make inferences about its environment. It’s essential for object recogntion, which must be capable of working independently of the visual frame of reference. However, though visual processing and object recognition are isolated to some degree and they are independent – they must also be interdependent.

Spidey Time

At some point, there is a temporal component induced into the visual processing system. Adding the temporal dimension significantly complicates the neural networks required, i think. And the degree to which temporal information components affect the structure of outer neurons is likely much greater than the degree to which the inner components are affected. Again, i’m refering to the neurons closest to the input as the outer neurons.

And so, this would mean that time – as experienced by the spider as opposed to time as signaled by neurons – is less tightly coupled to the higher-level inner components. I hope I just blew your mind again.

So this temporal component in the outer neural layers allows the spider’s subconscious networks to better infer how its actions and how the change in state of its environment will affect those networks’ “perception” of themselves … if that makes sense. In other words, the introduction of temporal components to neural networks, while significantly complicating them, also expand their functionality, while making them more efficient.

This underscores a very important point about the nature of information and knowledge:

The more information and knowledge you have, the easier it becomes to:

(1) acquire additional knowledge

(2) determine the accuracy of inferences based on your existing knowledge

(3) increase the efficiency of #1 and #2

That should both enlighten you and scare the fuck out of you, when taken in the context of generalized AI.

So, for a spider, it must compare its current visual input state with that of its memory of previous experiences: long term, short term and immediate short term. Instead of being divided into three distinct categories, this is moreso a continuoum of long term & short term.

So, for an example of immediate short term temporal processing, data must be fed back into the neural network. This would provide information on the spider’s motion with respect to it’s field of vision or about objects in motion.

Short and long-term information is likely fed back into the neural network at a higher level. Therefore, the “longer-term” the experiences are, the more likely this is encoded into “inner” neural networks. It would be moreso involved into higher level pattern recognition and processes. This concept could be extended into infinity – asi am wont to do – and i would make the assumption that the most important knowledge is that which transends time. But I think I skipped a few steps there, lulz.

So, the medium short-term processing could provide information on patterns of objects in motion. Some of these patterns would need to be specific for object types. That is, the spider needs to recognize that large objects moving quickly towards it are threatening. And small objects moving towards it might”e food.

Long term information is used to make inferences about higher order information, like patterns & behaviors of objects within it’s environment. Or behaviors, like feeding, web-making, etc.

Some of these higher-order patterns need to map to connections to phenomena imprinted by instinct. The phenomenon of food or fear. So there must be some mechanism whereby a spider innately recognizes large objects moving quickly towards it as dangerous, without the need of experience to induce a fearful response. I believe that instinct is moreso what causes the spider to more quickly correlate these experiences as being positive/negative. This is in contrast to the idea that instinct is specific knowledge encoded during gestation. So, IMO, instinct is a mechanism that enables some experiences to be categorized more quickly and to be imprinted more strongly.

Do Spiders Dream of Electric Sheep?

None of what I’m talking about is consciously processed, of course. I do believe that all animals share a capacity for consciousness. I’m making a ton of assumptions in this article, from an incredibly limited set of observations. This is all information that is represented by cellular networks of neurons, neurotransmitters and ions. I’m not saying the spider ‘thinks’ about this. Or that the spider somehow philosophizes about phenomena in its environment to make useful insights. I’m not sure if the automatic nature of this makes it more or less amazing.

I’m leaning towards more amazing. To me, it is simply astounding that such a small animal with a microscopic brain, virtually incapable of thought, is somehow capable of producing a such a vast spectrum of information about their environment. Their capacity for pattern recognition crucial to their survival, the degree to which it is efficent, and the immeasurable difference to our own capacity for AI – fascinating. Furthermore, they are capable of doing this using only input from their environment.

And with virtually zero communication: I say that because every interaction – whether using language or via non-typical interaction with another species – requires a certain degree of communication in the form of metainformation exchange. In other words, every interaction carries a baseline level of information – and it is critical – as a kind of meta-information. So, basically, the information ‘exchanged’ through one species input neurons to another species input neurons – it IS communication. It might not be symbolic communication. We might not recognize it as such, but it is.

And, for further astonishment, these cellular networks autonomously adapt to provide aggregate information based on inputs to each neural node. This reinforces encoded information that seems correlated to previously recognized patternr or has shown to be useful in predicting the state of the spider’s environment. And it must do this effectivly enough to avoid a fatal mistake before reproduction. These neural networks must be capable of combinating and recombinating all the input data.

Ad Nihilum

Anyway, as you can clearly see, a limited amount of information will often set my mind off on a chain reaction of inferences. It’s pretty fucking annoying honestly. I have lots of shit to do. What is frustrating is that this happens because i’m intelligent AND i don’t have the answers. There’s many subjects I spend an inordinant amount of time thinking about, where I wouldn’t have to expend the effort if I just knew this answers.

This underscores the ironinc dichotomy of becoming smart and becoming intelligent. College makes you smart. That’s the correct path, the best path for most people. But honestly, it is the constant and painful struggle which has made me intelligent. It is that I DO NOT KNOW, which makes me struggle to learn. And because I have struggled to learn on my own – and wasted so much time doing so – this is why I’m able to make such insights. However, if everyone took this path, our systems of knowledge propagation would crumble. And my own blind spots are glaring. It’s often pretty embarrassing what I don’t know.

For years, I thought white eggs were white because they went through a bleaching process … LMAO ok that’s embarrassing.

If I went to college, I would have these answers. The system of information processing and structure of knowledge in my mind would look just like everyone else’s. I wouldn’t have the need to search so hard. I also would lack my propensity for creativity, but the number of times i have ‘discovered’ something that already exists both amazes me and embarrasses me.

If I was smarter and if my mind embodied a more similar structure of knowledge to that of the college educated, I would be much more capable of communicating these ideas. Because I didn’t learn most of this in college, I tend to develop my own vocabulary to describe what I’m talking about. And because most college educated people are used to talking about their domain of expertise using a limited set of vocabulary, I often find myself getting tuned out, since I’m not using the right words and I don’t have the same understanding of distinctions between ideas – but what if some of those distinctions in the commonly understood structure of knowledge were inaccurate, misleading or limiting?

And finally, if I did excel in college – i fell through the cracks, truly I didn’t even try – then I would have been understood to be extremely smart. I would have been whisked away at some inordinately expensive project lasting years. I would narrowly focus on one subject and perhaps waste my life. I hate to say this, since – as much as I have issues with the academic system – I also believe it has great value and I do not want to demonstrate the wrong path for others.

Peace