Can a machine learning model predict the pattern of given sequence? I am curious if it is possible to do so.
For example, if I supply

$[0, 1, 2, 3, 4, 5]$, the model should return "natural number sequence",

$[1,3,5,7,9,11]$, it should return "natural number with step of $2$",

$[1,1,2,3,5]$, it should return "Fibonacci numbers",

$[1,4,9,16,25]$, it should return "square natural number"

and so on.

Those all fit into a single quadratic, auto-correlated model.

$$ x_0 = a \\ x_i = b i^2 + c x_{i-1} + d i + e $$

The sequences can be curve fitted producing a set of $n$ perfect fits of the form $(a, b, c, d, e)$ given the above model. A rules engine given the correct parameterized rules can produce the most desirable verbal description from among the $n$ fits in the set. The rules can also be prioritized by a simple feed forward network trained to simulate the most natural selection of string descriptions from any set of fits where $n > 1$.

This will work well for the examples in the question and many more, however, if the sequence $\{1, 4, 1, 5, 9\}$ is fed into the system, it will produce some weird description based on the quadratic, auto-correlated model it was given rather than, "digits of $\pi$ to the right of the decimal place."

The only way to produce the most common response a university freshman math student would produce would be to extend the boundaries of AI engineering first. For example, once an AI system is developed that can handle natural language and cognition like a child, several of them can be separately trained in a simulation of primary and secondary school mathematics. The median response for each sequence given to the class of AI students (class made up of artificial students studying math, not class of humans studying AI) will then be a reasonable prediction of what human university freshmen would produce as a median response.

This can be framed as a classification problem where a model is supervised on a dataset containing finite-length number sequences $x^{(i)}_1, \cdots, x^{(i)}_n$ and the sequence name $y_i$. For example, the dataset could look like this:

([0,1,2,3,4], 0)
([1,3,5,7,9], 1)
([1,1,2,3,5], 2)
([1,4,9,16,25], 3)

where the numbers on the right are integer representations of the sequence name. Given intuitive sequences, plentiful training data, and training examples of reasonable length, this problem would not be that difficult to solve. Sequence models from deep learning, such as recurrent networks (LSTM, GRU) or temporal convolutional networks, are well-suited for tasks such as this one.

Of course, this is only possible within certain constraints. The models are only good at what they are trained to do, so it would be impossible to use them to infer whether a sequence skips by 2 or 3 without having that information explicitly present in the training data. It would be interesting to see whether unsupervised models could detect this sort of information, although I don't think this work has been done in the present.


Is AI programming useful in everyday programs? I'm curious about Artificial Intelligence. In my everyday job I develop standard applications, like websites with basic functionalities like user subscription, file upload, forms saved in a database...

I mainly know of AI being used in games or robotics fields. But can it be useful in "standard" application development?

Yes, but probably only to a limited degree in the near term.

Where people draw the boundaries around 'artificial intelligence' is fuzzy, but if one takes the broad view, where it incorporates any sort of coding of explicitly cognitive functions, then many routine economic tasks can benefit from artificial intelligence. Many search engines, for example, can be seen as offering artificial intelligence applications as a service.

For more 'standard' applications, most near-team applications of AI have to deal with fraud detection and prevention. If you track a user's cursor moving across the screen, for example, you can build a model that differentiates between humans and bots, and treat the two separately. See this article for an example.

In the longer term, of course, a program that could write programs could write these sort of applications like any other.

AI or Artificial Intelligence

What is it?

Artificial intelligence (AI) is intelligence exhibited by machines. In computer science. Colloquially, the term "artificial intelligence" is applied when a machine mimics "cognitive" functions that humans associate with other human minds, such as "learning" and "problem solving".

Can it be useful in a "Standard" application?

Well, what I think about a Standard application using AI is that AI is used for that too, because when the machine have a reaction of the user input is AI or Artificial Intelligence. So the AI in Standard application it have been used many years ago already.

PS: If there are grammar errors, then I'm sorry because I'm not a English speaker.

Sources: https://en.wikipedia.org/wiki/Artificial_intelligence "AI or Artificial Intelligence."

DevJosueDav
Just a C# Artificial AI Intelligence Developer.


Do I need classification or regression to predict the availability of a user given some features? While studying data mining methods I have come to understand that there are two main categories:

Predictive methods:

Classification
Regression

Descriptive methods:

Clustering
Association rules

Since I want to predict the user availability (output) based on location, activity, battery level (input for the training model), I think it's obvious that I would choose "Predictive methods", but now I can't seem to choose between classification and regression. From what I understand this far, classification can solve my problem, because the output is "available" or "not available".

Can classification provide me with the probability (or likelihood) of the user being available or not available?

As in the output wouldn't just be 0 (not available) or 1 (for available), but it's be something like:

$80\%$ available
$20\%$ not available

Can this problem also be solved using regression?

I get that regression is used for continuous output (not just 0 or 1 outputs), but can't the output be the continuous value of the user availability (like the output being $80$ meaning user is $80\%$ available, implicitly the user is $20\%$ unavailable).

Yes. For instance, the popular softmax regression gives you probability distribution for each class.
Yes. Softmax is a regression over a set of discrete classes.

We can use regression for classification, the most common strategy is to grab the most likely class for the prediction.

You can use naive bayes classification and calculate posterior probabilities using prior beliefs or logistic regression can be used with sigmoid function.


Do GANs come under supervised learning or unsupervised learning? Do GANs come under supervised learning or unsupervised learning?

My guess is that they come under supervised learning, as we have labeled dataset of images, but I am not sure as there might be other aspects in GANs which might come into play in the determination of the class of algorithms GAN falls under.

The terms Supervised Learning and Unsupervised Learning predate the invention of the application of artificial networks to a generative and discriminative network pair, which was the first popular generative topology.  The existence of labeling is the key distinction between the two.  Even partial labeling indicates supervision, as odd as that jargon is, since the supervisor does no learning and the labels are constants.

Based on the original description of the discriminative network in a GAN, that it consumes examples without labels, GANs are a type of unsupervised learning.  That fact does not eliminate the use of labels as part of an extension of the original design that generates based on some labeled element in the examples or the use of other labels to indicate the fitness of each image for a class into which generated images are expected to fall.

GANs are unsupervised machine learning algorithms. According to Wikipedia, unsupervised algorithms are :

  Unsupervised learning is a branch of machine learning that learns from test data that has not been labelled, classified or categorized

In GAN networks, only training data is provided which is not labelled. The network generates candidates ( generative ) which are evaluated by the discriminator. The network slowly learns from the data given from a latent space.

  Suppose, you want to create a GAN network which can make a Monet painting. You just need to feed it some Monet paintings. Here, you are not interested in classifying the painting, but copying/mimicking it.

Hence, there is no need for labels here which makes GAN an unsupervised machine learning algorithm.


How do I perform object detection if there is only one type of object? How do I do object detection (or identify the location of an object) if there is only one kind of object, and they are more of less similar size, but the picture does not look like standard scenes (it is detection of drops on a substrate in microscopic images)?  Which software is good for it?

The hardest part is image annotation: here the difference between object recognition and object detection becomes important. If you just want to answer the question "does this image contain object X?", then you just need to provide as many images that contain object X as possible, together with as many images  that don't contain object X (but are otherwise similar). However if you want to answer the question "Where exactly object X is located in this image?" then you will need to manually provide a bounding box for each instance of object X in each image. Obviously, the second scenario is a lot more labor intensive.

After you've done this part, train either a binary image classifier (typically this will be a convolutional neural network) on your annotated images (split them into train and test partitions), or an object detector (googling "custom object detection" produces lots of code examples how to train it (e.g. https://towardsdatascience.com/tutorial-build-an-object-detection-system-using-yolo-9a930513643a start with Step 2B).

Such applications are usually dealt with by using some image processing and computer vision techniques. As far as I remember, we used Correlation in Digital Signal Processing using MATLAB last semester. The method was simple:
METHOD 1 : Using Cross Convolution
- Import the image you want to use
- Select the object in the image that you want to detect (a drop in your case) using its coordinates
- After some processing, search the image for the drop, using the selected image as reference.
(So, MATLAB actually calculates cross convolution of each coordinates of the actual image with that of the 'drop' image)
Read this for detailed method
This requires you to obtain MATLAB and a normal PC

If you can manage to get better hardware or more time (as slower processing demands more time), you can opt for method 2
METHOD 2 : Using Deep Learning
- Download loads of images of drops or the object you are looking for
- Train the model and create a Convolution Neural Network (Hardware and time intensive)
- Validate the network and test it
(this method is more widely used and is a offspring of Artificial Intelligence/ Machine Learning . It provides a far more accurate result at the cost of the images, hardware and time you spend on training the model. Nowadays, people use Python for the same purpose as it saves you from the new syntax that you might face using MATLAB. Both are more or the same equally efficient)
Read detailed methodology here and here


What does the argmax of the expectation of the log likelihood mean? What does the following equation mean? What does each part of the formula represent or mean?

$$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$

This equation and more information of it can be found in Expectation Maximization Wikipedia site and the explanation there was as follows (formula there in two parts):

Some more explanation from same page:

  In statistics, an expectation–maximization (EM) algorithm is an iterative method to find maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

Mathematically, E in your equation stands for Expectation Value,  x|theta is conditional probability and x~data and model are sub-titles of source of probability in either case. The arg max theta is argument theta that maximizes the equation.

I am correct in thinking that you would like an intuitive feel or explanation?

After thinking about this for a little bit, I thought I would approach this problem from understanding the K-means algorithm. As you may know the k-means algorithm uses some straight forward calculations. The computer calculates the distance from one point to a center. If you can measure the distance of a point (x1) to center 1 and from point (x1) to center 2 then you are golden. Because next all you do is choose the minimum argument or distance and that becomes the group that the point will belong to.

So, in our example of K-means, let us say a human picks the number of groupings, k=2. Meaning the human thinks there are only two types of 'things'. Therefore all the things, items or whatever that are closer to center 1 are called '1'. The items that are closer to center 2 are now called '2'.

In our K-means case it chooses the distance which is the shortest. It chooses the minimum value of distance and assigns the future points to '1' or '2' (for example).

Well, what if you don't want to use distance as your measure but instead you just learned about Gaussian curves and what to call your items as being from Gaussian curve '1' or '2'. We could do that, right? Well, if I used Gaussian curves as my measuring criteria, I could use probability instead of distance (as we did in K-means).

Now let's say we started to look at our points in terms of what is the probability of point 1 being related to center '1'. Let's say we knew the mean and standard deviation of our center '1'. Well we could then use probability to use as our new measuring stick now. So if point x1 is 1 standard deviation away from center '1' its p-value would be (let's say) (P of 1 std.dev.) = ~0.35.

Now what if we also knew the mean and variance of the second center, '2'. We could then calculate the Probability of our point x1 being part of center '2'.  Let's say that point x1 is 4 standard deviations from center '2'. It's p-value = 0.0002.

Therefore I would choose the MAXIMUM p-values to use to assign my points to my centers, NOT the Minimum that I used in K-means.

Really K-means is similar to E.M. except it uses distance not p-values.
Does that help?
P.S. Forgive the grammar mistakes, etc I am doing this late for me, lol.


How to figure out which words have the same meaning in two different languages? Imagine two languages that have only these words:

Man = 1,
deer = 2,
eat = 3,
grass = 4

And you would form all sentences possible from these words:

Man eats deer.
Deer eats grass.
Man eats.
Deer eats.

German:

Mensch = 5,
Gras = 6,
isst = 7,
Hirsch = 8

Possible german sentences:

Mensch isst Hirsch.
Hirsch isst Gras.
Mensch isst.
Hirsch isst.

How would you write a program that would figure out which words have the same meaning in English and German?

It is possible.

All words get their meaning from the information in which sentences they can be used. The connection with other words defines their meaning.

We need to write a program that would recognize that a word is connected to other words in the same way in both languages. Then it would know those two words must have the same meaning.

If we take the word "deer" (2) it has this structure in English

1-3-2
2-3-4

In german (8):

5-6-8
8-6-7

We get the same structure (pattern) in both languages: both 8 and 2 lie in first and last position, and the middle word is the same in both languages, the other word is different in both languages. So we can conclude that 8=2 because both elements are connected with other elements the same way.

Maybe we just need to write a very good program for recognizing analogies and we will be on the right track to creating AI?

You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools.

  We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning.

You are describing the essence of known natural language processing (NLP) tasks such as word alignment (link words in different languages that have the same meaning) and, of course, machine translation.

While learning a machine translation model, we actually do discover which words (or parts of words, or sequences of words) in different languages have the same meaning.

Here are some concepts I would recommend for further study of this subject:

Word alignment, an example for a well-known and popular tool would be fast_align
Word embeddings, word2vec is a widely used tool
Modern machine translation with sequence-to-sequence models, well-known tools are fairseq, or Sockeye

For this example the function below will do:

TSAI.Analogies.FindAnalogy(List ex1, List ex2, List ex3, out List ex4)

ex1 is to ex2 as ex3 is to ex4.
Figure out ex4.

Fill ex4 with values from ex2.
For every value in ex3: find out to which positions in ex4 we have to copy this value, based on value in ex1 at the same position that was repeated in ex2.


Which predictive algorithm can be used to predict a number given other numbers? I am currently searching for a supervised learning algorithm that can be used to predict the output given a large enough training set.

Here's a simple example. Suppose the training dataset is {[A=1, B=330, C=1358.238902], result=234.244378} and the test dataset {[A=893, B=34, C=293], result=?}

My intention is to predict ? using the input values and result given in the training dataset.

What algorithm would be effective for this problem given the wide range of my input/output values? Would this require some sort of regression algorithm?

Impossible to solve until you define an error measurement ( by example $|R-R'|$ or $(R-R')^2$ ) and how this error changes when A, B and C change.
Extreme example: assume $R()$ is random (unrelated to A, B, C values) but static (always same $R(A,B,C)$ for same values of A,B,C). Given some values of A, B, C, you can only answer the value of $R(A,B,C)$ when A,B,C was in the training set. $R(A,B,C)$ is undefined and no predictable when A,B,C was not in the training set.
Moreover, improvements can be done if $R()$ has some properties, by example, if it is possible to state that $R(A,B,C)=R(B,A,C)$ or that $R(A1,B1,C1)=R(A2,B2,C2)$ if $A1+B1+C1=A2+B2+C2$.

Without seeing more data it's hard to say for sure.  Superficially, this looks like a regression type problem.  As you mention, there is a lot of spread on the input values, but that doesn't necessarily mean that something like linear regression wouldn't work. Try it and see what kind of coefficient of correlation you get.  If it's really low, you probably need a different approach, OR the data might actually not have any (or much) predictive power in this scenario.

Beyond linear regression, you might find that there's a more complicated mathematical relationship between the inputs and outputs, that could be determined using symbolic regression.  Another possibility, if there is a complex non-linear relationship in play, is that an artificial neural network approach might work well.


How widely accepted is the definition of intelligence by Marcus Hutter & Shane Legg? I came across several papers by M. Hutter & S. Legg.
Especially this one:
Universal Intelligence: A Definition of Machine Intelligence, Shane Legg, Marcus Hutter
Given that it was published back in 2007, how much recognition or agreement has it received?
Has any other work better formalizing the idea of intelligence been done since?
What is considered current standard on the topic in the field?

My sense is that everyone is pretending Intelligence doesn't have a grounded definition, from which all other definitions arise:

Intelligence is a measure of utility in an action space μ(υ)

It can be a relative measure, in relation to other rational agents, or absolute in relation to solved games (problems).  An action space is any context, and formalized as problems or sets of problems, typically grouped by complexity class.
Hutter and Legg is an explication of this grounded definition which accounts for unlimited contexts (complexity classes/environments) and for increasing utility of a given agent over time (learning/optimization.)  Intelligence itself does not require learning or general applicability, but Hutter & Legg does not refute this, merely grades static intelligence and narrow intelligence as more limited.
Even this is subject to context, as more limited rationality can be more optimal.

The definition of intelligence is grounded because, while the term "intelligence" is a symbol, intelligence itself is function, the strength of which evaluated by a measurement of some result (utility)

It doesn't require defining the function to understand it as a function:  measurement of a result requires mechanism and decision/action.
You will find this natural language definition applies to even emotional intelligence, which relates to the observational capability of the rational agent in context, and allow that rational agent to make more optimal decisions in context.
This is similar in spirit to truth, only grounded in a formal logic context, where it is a condition and result, not an assertion.  By contrast, the the conditionality of truth is often obscured in a natural language context, where is is routinely applied to unvalidatable informal statements, and can even be conflated with the statement itself à la: "this is the truth!"

Just a few commonsensical remarks about why this kind of intelligence definition seems unable to capture the logic of life:

Optimization only makes sense in a stationary environment.  When many agents learn and interact, they are building a constantly changing environment.

Survival and reproduction is the only thing that really matters, and it does not require optimization, just good enough solutions.

The survival of individual living organisms heavily depends on adequate hardwired sensory abilities that can slowly change throughout many generations. But smarter individuals can use their brains (or whatever tools they may have endowed with plasticity) to quickly learn and adapt in non-stationary environments. Fast learning, not optimization, is what these agents really need.


Does gradient descent in deep learning assume a smooth fitness landscape? I've come across the concept of fitness landscape before and, in my understanding, a smooth fitness landscape is one where the algorithm can converge on the global optimum through incremental movements or iterations across the landscape.
My question is: Does deep learning assume that the fitness landscape on which the gradient descent occurs is a smooth one? If so, is it a valid assumption?
Most of the graphical representations I have seen of gradient descent show a smooth landscape.
This Wikipedia page describes the fitness landscape.

I'm going to take the fitness landscape to be the graph of the loss function, $\mathcal{G} = \{\left(\theta, L(\theta)\right) : \theta \in \mathbb{R}^n\}$, where $\theta$ parameterises the network (i.e. it is the weights and biases) and $L$ is a given loss function; in other words, the surface you would get by plotting the loss function against its parameters.
We always assume the loss function is differentiable in order to do backpropagation, which means at the very least the loss function is smooth enough to be continuous, but in principle it may not be infinitely differentiable1.
You talk about using gradient descent to find the global minimiser. In general this is not possible: many functions have local minimisers which are not global minimisers. For an example, you could plot $y = x^2 \sin(1/x^2)$: of course the situation is similar, if harder to visualise, in higher dimensions. A certain class of functions known as convex functions satisfy the property that any local minimiser is a global minimiser. Unfortunately, the loss function of a neural network is rarely convex.
For some interesting pictures, see Visualizing the Loss Landscape of Neural Nets by Li et al.

1 For a more detailed discussion on continuity and differentiability, any good text on mathematical analysis will do, for example Rudin's Principles of Mathematical Analysis. In general, any function $f$ that is differentiable on some interval is also continuous, but it need not be twice differentiable, i.e. $f''$ need not exist.

Does deep learning assume that the fitness landscape on which the gradient descent occurs is a smooth one?
One can interpret this question from a formal-mathematical standpoint and from a more "intuitively-practical" standpoint.
From the formal point of view, smoothness is the requirement that the function is continuous with continuous first derivatives. And this assumption is quite often not true in lots of applications - mostly because of the widespread use of ReLU activation function - it is not differentiable at zero.
From the practical point of view, though, by "smoothness" we mean that the function's "landscape" does not have a lot of sharp jumps and edges like that:

Practically, there's not much difference between having a discontinuous derivative and having derivatives making very sharp jumps.
And again, the answer is no - the loss function landscape is extremely spiky with lots of sharp edges - the picture above is an example of an actual loss function landscape.
But... why the gradient descent works then?
As far as I know, this is a subject of an ongoing discussion in the community. There are different takes and some conflicting viewpoints that are still subject of a debate.
My opinion is that, fundamentally, the idea that we need it to converge to the global optimum is a flawed one. Neural networks was shown to have enough capacity to completely remember the training dataset. A neural network, that completely remembered the training data. has reached the global optimization minimum (given only the training data). We are not interested in such overtrained models - we want models that generalize well.
As far as I know, there is no conclusive results on which properties of the minimum are linked to ability to generalize. People argued that these should be the "flat" minima, but then it was refuted. After that a "wide optimium" term was introduced and gave rise of an interesting technique of Stochastic Weight Averaging.


What sort of game problems can neural networks trained/evolved with evolutionary algorithms solve, and how are they typically implemented? I'm interested mostly in the application of AI in gaming; in case this adjusts the way you answer, but general answers are more than welcome as well.
I was reading up on Neural Networks and combining them with Genetic Algorithms; my high-level understanding is that the Neural Networks are used to produce a result from the inputs, and the Genetic Algorithm is employed to constantly adjust the weights in the Neural Network until a good answer is found.
The concept of a Genetic Algorithm randomly mutating the weights on the inputs to a Neural Network makes sense to me, but I don't understand where this would be applied with respect to gaming.
For example, if I had some simple enemy AI that I want to have adapted to the player's play-style, is this a good opportunity to implement the AI as a Genetic-Algorithm combined with a Neural Network?
With these different suitable applications, how does one go about deciding how to encode the problem in such a way that it can be mutated by the Genetic Algorithm and serve as suitable on/off inputs to a Neural Network (actually, are Neural Networks always designed as on-off signals?)?

Without going in too much detail on how exactly Neural Networks and Generic Algorithms work, I can tell you that both the algorithms are not good candidates for computer games.  They work well in scientific environments where the system is "trained" on a huge data set to adjust the "weights" (variables) for a given problem.  This "training" process requires a lot of processing power, time and a large data set.

Computer games, however either needs to run in real-time (no time for training) or turn-based (not enough data for training).

Another problem is that computer games need to free up as much as possible system resources for physics, graphics, sounds and the user interface to improve the player's experience so game developers usually use other lighter techniques (like a rule-based system) to create the illusion of an AI player.

"if I had some simple enemy AI that I want to have adapt to the players play-style, is this a good opportunity to implement the AI as a Genetic-Algorithm combined with a Neural Network"

Sure. Just provide a quality measure for the GA that's related in some manner to the effect of the player's actions on the game state/opponent(s).

For example, if defining an opponent's intelligence, one of the conceptually simplest things would be to give a GA population member a fitness that's inversely proportional to the increase in the player's score over some period of time.

  are Neural Networks always designed as on off signals?)?

No. In general, they can be considered to perform nonlinear regression, i.e. a mapping from a vector of real numbers of length n to another of length m. Classification (i.e. 0/1 outputs can be seen as a restricted case of this).

As per my answer to this AI SE question, there is a large body of literature (and mature software libraries) for using evolutionary computation to encode neural nets.

More generally, some early work in 'online adaptivity using GA-encoded NNs' appeared in the Creatures http://mrl.snu.ac.kr/courses/CourseSyntheticCharacter/grand96creatures.pdf series of games by Steve Grand (details).


Measuring Soft Skill How can we attach numerical value to the measurement of soft skill? Is there any book, site or materials that can teach me how to do this?

I want to know if there is any way we can measure soft skill as listed  here and say a person  has (over 100) 70% or 60% in critical thinking ability or his critical thinking ability is over 72% and be accurate with the score.

I spent some time on the Future of Work in UK government. If you want to look at the impact of AI on work you need to have good definitions of soft skills that you can measure and you can track progress against.

So the definitions sometimes need to change and decomposed. Take "Coaching" from that list: there are now bots (such as Wysa.ai) that coach you. You can measure coaching as a skill by the effectiveness of the participants achieving goals.

Things really get murky when you talk about "Creativity" -- I wrote a piece on AI impact of creativity in a similar vein.

"Soft skill" is assessed by our common sense and there is still no standardized way of assigning soft skill , however good IQ is often a good indicator of social intelligence and soft skill as well (except for savant cases).


Do we have to use CNN for Deep Q Learning? I read top articles on Google Search about Deep Q-Learning:

https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8
https://skymind.ai/wiki/deep-reinforcement-learning
https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/
Q-Learning page on wikipedia*

and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term "Deep Q-Learning" on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.

*it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning

No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: github.com/keon/deep-q-learning/blob/master/dqn.py

DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now.

The approximator can be any artificial neural network architecture, including deep fully-connected networks.


Representation of real numbers in Genetic Algorithm Take a look at section 2.2.2 of this book (from Page-15 to 16).

  2.2.2 Representation and Evaluation

  $$max f (x)= x sin(10πx)+2.0 ... ... ... (2.8)$$
  $$s.t. −1 ≤ x ≤ 2$$

  We can use a real number, in the range $[−1,2]$, to represent a solution in Eq. $2.8$ directly. Many operators can handle real number representation. But we use the binary code or binary representation here for two reasons. GAs were originally proposed to be binary code to imitate the genetic encoding of natural organisms. On the other hand, binary code is good for pedagogy. A binary chromosome is necessary to represent a solution x in the scale $[−1,2]$. The same holds for the binary representation of real numbers in a computer.

  In binary code, we cannot represent a real number completely correctly, so a
  trade-off is necessary. A tolerance needs to be deﬁned by the user, which means the errors below the tolerance are extraneous. If we divide the deﬁnition domain into $2^1 = 2$ parts evenly and select the smallest number in the parts to represent any number in the division, we can only represent $−1$ and $0.5$ by $0$ and $1$ respectively. $2^2 = 4$ divisions make the $00$, $01$, $10$, and $11$ represent $−1$, $−0.25$, $0.5$, and $1.25$, respectively. The larger division number we select, the less error there is in representing a real number on binary code. Suppose we use $100$ binary codes to represent a real number in the range $[−1,2]$; the maximum error is $\frac{3}{2100} ≈ {2.37}^{−30}$, which would be satisfactory for most users. In this way, we can represent a real number with any accuracy requirements.

  In this problem, we use $l = 12$ binary codes to represent one real number as follows, which constitutes a chromosome to be evolved.

Actually, I haven't understood this text. I know that binary numbers are already able to represent fractions.

So, what are they talking about?

The authors want to use a binary genome which contains specific interpretation/mapping to a fixed precision real number. They could use one or more existing schemes, but these typically won't represent the exact precision and range of real numbers they want to use in the example problem.

They use their own encoding scheme and explain it in the text for completeness. Any similarities between their scheme and other binary representations of real numbers are to be expected, and if you understand them, it will help understand what they are doing here too.

GAs don't have to use this sort of scheme for real numbers. Often they use standard floating-point representation as separate elements. But the authors want a binary code here to help teach some basic concepts, which is what they mean by "on the other hand, binary code is good for pedagogy"

Discrete Experiments

The first impression is that real numbers (or IEEE floating point approximations of them) don't belong in genetic algorithms because DNA genetics is based on  binary representations based on the presence of either adenine-thymine or cytosine-guanine in a position along a chromosome. Sub-sequences of those bits correspond to specific triggers and resulting protein syntheses. Genetic information is discrete, not continuous.

That is true, but, as the number of instances being represented probabilistically approach infinity, the distance between rational numbers that represent the potential probability values tend toward zero. That is, where $\mathbb{I}$ is the set of indices for all possible probabilities for a given discrete scenario, we have this limit that tends toward but never reaches continuity.

$$ \forall \, j \in \mathbb{I}, \, \lim_{n \rightarrow \infty} P_j - P_{j-1} = 0 $$

Also, as the number of bits in the mantissa of a fixed point number increases, the distance between their values similarly decrease toward the limit of zero.

When the authors state, "Binary code is good for pedagogy," they probably mean that it is difficult to store a voltage without sampling it and representing it as an integer that can be stored in noise and drift resistant ways using binary encoding and error correction strategies. If this is a correct expansion of what that phrase means, then it makes much sense. In analog circuitry, a voltage may be latched for a few microseconds while it is sampled, capacitor leakage is too high to trust the value much longer.

Furthermore, far more samples are often taken than what is prescribed by the Nyquist criteria so that noise cancellation improves the signal to noise ratio. Once the signal is in digital form, error correction and detection reduces these phenomena significantly. Although we can prove this using mathematics, the hyper-evolution of the evolutionary mechanism itself has proven these facts over a few billion years.

At this point, the theory presented from the authors is clear. They are stating three things.

Real numbers represented in computers as integers, IEEE floating point numbers, or arbitrary precision numbers using appropriate libraries are discrete approximations of continuous values.
DNA represent in binary state-action relationships the action plans based on continuous probability distributions in the environment with which the system interacts.
These two facts create a fundamental correlation between genetic information and binary approximations of real numbers in computers.

Although the first two statements are correct, the third is not. This is the kind of fundamental error common in statements made by people with excellence in mathematics and computer science but somewhat less profoundly educated in the nuances of biology and probability. Consider these discrete events that affect the probabilities that some enzyme or protein should be generated under some particular threshold condition in the cellular machinery.

A lion kills a jackal of two
One of the two had a child jackal
The other did not
A particular combination of AT versus CG bits led to the manufacture of a protein that allowed a faster muscular response, which allowed the remaining jackal to propagate

The function representing the event of capture is not continuous. The probability of a death event between birth and conception of the child jackal is effectively continuous. For a child jackal, the probabilities of their conception is irrelevant. The pertinent function from the perspective of that specific line of jackals outputs exactly one bit. Zero is nonexistence and one is existence.

The function that may model evolution surrounding this scenario is a continuous appearing function, but it is made up of discrete events that may be conditioned by real numbers but, since injured jackals can still reproduce, do not result in real number outcomes. This is where the authors are not accurately modeling biological reality. A functional representation of selection on the basis of a single genetic bit is

$$ S_i = S(E, P(G_i, T)) \, \text{,} $$

where the survival bit $S_i$ for a given offspring is an effectively randomized selection from the set $\{0, 1\}$, given the environmental scenario in which the parent lived from birth to the spawning of an offspring $E$, and the probability of a phenotype $P$, which is a function of the set of genetic bits in the genotype $G_i$ (the bits in question), and the triggers $T$ of the expression of that genotype presented by both internal and external organism state.

Here is where the authors clearly diverge from what is known about the genetic machinery in the context of adaptation of a species.

  We can use both 101001101011 and 0.9534 to represent an individual (or a chromosome); the former is called a genotype representation of an individual and the latter a phenotype representation of an individual.

The items that lead to the event is not a single real number but a set of discrete genotypes. To encode them as a real number is to obfuscate the discrete separations between multiple discrete genotypes that impact any given risk event, such as a lion, the attack of which can potentially eliminate the genotypes and the entire DNA strand of which they are a part. Neither a real number nor its digital approximation would be a straightforward and efficient representation of separate genetic characteristics that may impact a large number of genetic expressions, protein manufacturing runs, phenotypes, more continuous physical traits, and probabilistic advantages related to risks to the spawning of offspring.

Therefore, the confusion expressed in the question may be founded in a legitimate sense of a flaw in the logic of the presented material. Of course, that does not discount the possible expertise and legitimacy of other information presented in the same text.


What are some information processing models besides feedforward or multi-layered neural networks? Feedforward or multilayered neural networks, like the one in the image above, are usually characterized by the fact that all weighted connections can be represented as a continuous real number. Furthermore, each node in a layer is connected to every other node in the previous and successive layers.

Are there any other information processing models other than FFNNs or MLPs? For example, is there any system in which the topology of a neural network is variable? Or a system in which the connections between nodes are not real numbers?

Neural Network equivalents that is not (vanilla) feed forward Neural Nets:

Neural net structures such as Recurrent Neural Nets (RNNs) and Convolutional Neural Nets (CNNs), and different architectures within those are good examples.

Examples of different architectures within RNNs would would be: Long Short Term Memory (LSTM) or Gated Recurrent Unit (GRU). Both of these are well described in Colah's blog post on Understanding LSTMs

What are some alternative information processing system beside neural network

There are sooo many structures. From the top of my head: (Restricted) Boltzmann machine, auto encoders, monte carlo method and radial basis networks to name a few.

You can check out Goodfellow's Deep learning-book that is free online and get the gist of all the structures I mentioned here (most parts requires a bit of math knowledge, but he also writes about them quite intuitively).

For Recurrent Neural Nets I recommend Colah's blog post on Understanding LSTMs

Is there any system in which the topology of a neural network is variable?

Depends on what you mean with the topology of a neural network:

I think in the common meaning of topology when talking about Neural Networks is the way in which neurons are connected to form a network, varying in structure as it runs and learns. If this is what you men then the answer, in short, is yes. In multiple ways actually. On the other hand, if you mean in the mathematical sense, this answer would become a book that I wouldn't feel confortable writing. So I'll assume you mean the first.

We often do "regularization", both on vanilla NN and other structures. One of these regularization techniques are called dropout, which would randomly remove connections from the network as it is training (to prevent something called overfitting, which I'm not gonna go into in this post).

Another example for another way would be on the Recurrent Neural Network. They deal with time series, and are equipped for dealing with timeseries of different lengths (thus, "varying structure").

Does it exist neural net systems where complex numbers are used?

Yes, there are many papers on complex number machine learning structures. A quick google should give you loads of results. For example: DeepMind has a paper on
Associative Long Short-Term Memory which explores the use of complex values for an "associative memory".

Links:

Goodfellow's Deep Learning-book: deeplearningbook.org

Colah's blogpost on RNN's: colah.github.io

Paper on DeepMinds Associative LSTM: arxiv:1602.03032

A very popular choice are Hidden Markov Models.


What are good alternatives to the expression "Artificial Intelligence"? I read a really interesting article titled "Stop Calling it Artificial Intelligence" that made a compelling critique of the name "Artificial Intelligence".

The word intelligence is so broad that it's hard to say whether "Artificial Intelligence" is really intelligent. Artificial Intelligence, therefore, tends to be misinterpreted as replicating human intelligence, which isn't actually what Artificial Intelligence is.
Artificial Intelligence isn't really "artificial". Artificial implies a fake imitation of something, which isn't exactly what artificial intelligence is.

What are good alternatives to the expression "Artificial Intelligence"? (Good answers won't list names at random; they'll give a rationale for why their alternative name is a good one.)

Artificial is said to derive from the Latin word "artificium" which connotes ideas such as crafting.  Thus, artificial is a correct usage, and algorithms can be regarded as "artifacts" in the context of information as opposed to physical manifestation of information (i.e. matter).

However, I agree that the use of artificial is problematic in that, should strong Artificial General Intelligence ever be achieved, there is a stigma to "artificiality" that could have implications regarding personhood.

My personal feeling is that we should be using:

Algorithmic Intelligence

which this is functional definition, and therefore more meaningful than "artificial".  Additionally, "algorithmic" is a neutral term, and provides a very accurate description of what these systems are.

In terms of what is considered "intelligent", you may want to look at the concept of Bounded Rationality.  There is no hard definition of "intelligence", just degrees of optimality in regard to decision making in a condition of uncertainty.

Because this is subjective for any problem that is not solved, modifiers are utilized, and thus we refer to AI as "strong" or "weak". These terms are also used to describe the degree to which certain types of problems (for instance a non-chance, perfect information game like Checkers) has been solved. Complexity theory will shed more light on this concept.

For more insight on "artificial", you might find this question on the philosophical origin of the Turing Test interesting, because it partly involves the meaning of a "thing". (There were multiple words for this in Ancient Greek.)

Machine Intelligence

I believe intelligence is not a proprietary entity meant only living beings.

In fact the very origin of human intelligence is unknown. It is still not known if we can generate a brain just by fixing up the corresponding molecules of the real brain(Even theoretically). Even if we could do that does that constitute a real intelligence or artificial intelligence is still very hazy.

I think the phrase "Machine Intelligence" would sound appropriate.


If mankind can create artificial life in a machine, when would we define it's death? Can one actually kill a machine? Not only do we have problems in defining life, we also have problems in defining death. Will this also be true in artificial life and artificial intelligence?

Death as we know it for natural life is terminal. That is once dead, natural life cannot come back (at least in the current understanding and with current technologies---some people believe otherwise).

Death for AI is trickier. There may be only one scenario: Global destruction: Extreme scenario where everything supporting the existence of an AI disappears. This is equivalent to death in natural life, and low probability. It means all AIs die at once (as well as us).

We also do not know the degree and form of embodiment necessary for AGIs. We can assume now that hardware is replaceable indefinitely, thus "limiting" death to the above extreme scenario. But AGIs "body" may not be indefinitely replaceable. Then a definition closer to natural life death may be necessary.

We see arguments for two other scenarios, that I refute below:

"Static Death": An AI is still "defined" or "saved" somewhere (whatever it means actually), but it is not authorized or able to use resources. Assuming an AI is made of hardware and software, it is like a program stored on a disk, but without permission to run.

"Dynamic Death": Under the same characterization of AI as hardware and software, dynamic death is the invalidation of progress akin to strong liveness properties, where an AI is trapped in an infinite loop (or a void loop), in a form of "active death", as what happens to Sisyphus in Greek mythology. This is different from static death, as the AI still uses dynamic resources, although it cannot make progress. Continuing under the same assumptions, such AI could be "loaded" in main memory, or locked waiting for inputs or outputs to complete.

Note that in these two scenarios, rebirth is possible, and they also subsume that there is an entity that can decide conditions for rebirth, or preventing it completely. Would this entity be an "admin", a god, other AIs, or a human is another question, really.

The terms "death" and "rebirth" here could just be changed for "imprisoning", where the dynamic version would be like our human prisons, and the static version would be like SciFi cryogeny. This is a bit of a stretch, but we can see an equivalence, and no good reason to qualify these two scenarios as deaths.

In conclusion, death for AI seems to be an exceptional, singular scenario, so AI cannot die in practice, except if we are wrong on how we think we can make AGIs. AI can however be imprisoned forever.

Note: The terminology above is completely made-up for the post. I do not have citations to back some claims, but it is based on readings and personal work (including in software verification).

There is no reason to treat hard AI different then humans.
Some people telling that you can make a snapshot of AI but there is no reason to not make a human snapshot also. We dont have technology for that but there is no any magical barrier that would make it impossible (save all biological data and then print your copy somewhere else. Why not?).

Its to early to talk about this as we do not understand our existence (Death term for biological creatures evolving all the time).

I bet that in the future we will merge with AI and the only question will be what death means for any intelligent existence.


What are the domains where SVMs are still state-of-the-art? It seems that deep neural networks and other neural network based models are dominating many current areas like computer vision, object classification, reinforcement learning, etc.

Are there domains where SVMs (or other models) are still producing state-of-the-art results?

State-of-the-art is a tough bar, because it's not clear how it should be measured. An alternative criteria, which is akin to state-of-the-art, is to ask when you might prefer to try an SVM.

SVMs have several advantages:

Through the kernel trick, the runtime of an SVM does not increase significantly if you want to learn patterns over many non-linear combinations of features, rather than the original feature set. In contrast, a more modern approach like a deep neural network will need to get deeper or wider to model the same patterns, which will increase its training time.
SVMs have an inherent bias towards picking "conservative" hypotheses, that are less likely to overfit the data, because they try to find maximum margin hypotheses. In some sense, they "bake-in" Occam's razor.
SVMs have only two hyperparameters (the choice of kernel and the regularization constant), so they are very easy to tune to specific problems. It is usually sufficient to tune them by performing a simple grid-search through the parameter space, which can be done automatically.

SVMs also have some disadvantages:

SVMs have a runtime that scales cubically in the number of datapoints you want to train on (i.e. $O(n^3)$ runtime)1. This does not compare well with, say, a typical training approach for a deep neural network which runs in $O(w*n*e)$ time, where $n$ is the number of data points, $e$ is the number of training epochs, and $w$ is the number of weights in the network. Generally $w, e << n$.
To make use of the Kernel trick, SVMs cache a value for the kernelized "distance" between any two pairs of points. This means they need $O(n^2)$ memory. This is far, far, more trouble than the cubic runtime on most real-world sets. More than a few thousand datapoints will leave most modern servers thrashing, which increases effective runtime by several orders of magnitude. Together with point 1, this means SVMs will tend to become unworkably slow for sets beyond maybe 5,000-10,000 datapoints, at the upper limit.

All of these factors point to SVMs being relevant for exactly one use case: small datasets where the target pattern is thought, apriori, to be some regular, but highly non-linear, function of a large number of features. This use case actually arises fairly often. A recent example application where I found SVMs to be a natural approach was building predictive models for a target function that was known to be the result of interactions between pairs of features (specifically, communications between pairs of agents). An SVM with a quadratic kernel could therefore efficiently learn conservative, reasonable, guesses.

1 There are approximate algorithms that will solve the SVM faster than this, as noted in the other answers.

For datasets of low-dimensional tabular data. DNN are not efficient on low-dimensional input because of huge overparametrisation. So even if dataset is huge in size, but each sample is low-dimensional SVM would beat DNN.

More generally if data is tabular and the correlation between fields of the sample is weak and noisy, SVM may still beat DNN even for high dimensional data, but that depend on specific of data.

Unfortunately I can't recall any specific papers on subject, so it's mostly common sense reasoning, you don't have to trust it.


Can the addition of unnoticeable noise to images be used to create subliminals? I was reading this report: https://www.theverge.com/2017/4/12/15271874/ai-adversarial-images-fooling-attacks-artificial-intelligence

Researchers used noise to trick machine learning algorithms to misidentify or misclassify an image of a fish as a cat. I was wondering if something like that can be used to create subliminals.

What I mean by subliminals: United Nations has defined subliminal messages as perceiving messages without being aware of them, it is unconscious perception, or perception without awareness. Like you may be aware of a message but cannot consciously perceive that message in the form of text, etc.

All the reports about the noise trick said the noise was so transparent that humans couldn't detect it. This can be changed to make it noticeable unconsciously but unnoticeable at a conscious level so a human can register the subliminal but not be aware of it.

Is it possible to take an output from a hidden layer to construct such subliminal for humans, with trial and error one can find right combination? Can it be possible to come up with a pixel pattern or noise with ML which allows one to impose subliminals?

I would say no, at least not now. To achieve that you need to have a model that represents the structure of a human brain at a very detailed level, so that inherits the same flaws.


Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games? I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e. 20% of moves are random.

In all configurations, I initiate the weights to random values.

What I've found is that in all configurations, Player 1 will win the majority of games, or Player 2 will. There's no 50/50 split.

Is this expected or normal? I am training on the Connect Four game.

From wikipedia article on  Connect Four:

  Connect Four is a solved game. The first player can always win by
  playing the right moves.

It's pretty reasonable that even without training 1st player win more often by small margin.
With some training  1st player should mostly win.


Is it possible to prove that the target policy is better than the behavioural policy based on learned Q values? I have retrospective data for a sort of "behaviour policy" which I will use to train a deep q network to learn a target greedy policy. After learning the Q values for this target policy, can we make the conclusion that because the Q value for the target policy, $Q(s,\pi_e(s))$ is higher than the Q values for the behaviour policy, $Q(s,\pi_b(s))$ at all states encountered, where $\pi_e$ is the policy output by deep Q-learning and $\pi_b$ is the behaviour policy, then this target policy has better performance than the behaviour policy?

I know the proper way is to run the policy and do an empirical comparison of some sort. However, that is not possible in my case.

No, mainly because these are all stochastic approximations and may not represent the true values.

Almost nothing good can be said about NN approximations to value and Q functions(at least according to a professor I have had).


How do I classify an image that contains only polygons? I have two closed polygons, drawn as connected straight black lines on a white background. I need to classify such images in to three forms

Two separate polygons
One polygon encloses the other
The two polygons overlap each other.

The polygons vary in sizes and location on the image, and the image contains only the polygons and the white background.

Which neural network architecture should I use to solve this problem?

To make it easy take small known CNN network like Alex Net, train like : input  is image, output is [1,0,0,] separate, [0,1,0], encloses, [0,0,1] overlap. Cause you task is easy i guess that will be fine. But it can be done without ML just analysing image with constant algorithm by where are the points of poligons.


How to show Sauer's Lemma when the inequalities are strict or they are equalities? I have the following homework.

  We proved Sauer's lemma by proving that for every class $H$ of ﬁnite VC-dimension $d$, and every subset $A$ of the domain,

  $$
\left|\mathcal{H}_{A}\right| \leq |\{B \subseteq A: \mathcal{H} \text { shatters } B\} | \leq \sum_{i=0}^{d}\left(\begin{array}{c}{|A|} \\ {i}\end{array}\right)
$$

  Show that there are cases in which the previous two inequalities are strict (namely, the $\leq$ can be replaced by $<$) and cases in which they can be replaced by equalities. Demonstrate all four combinations of $=$ and $<$.

How can I solve this problem?

I am not good with math notations so I'll stick to language more.

The first equality holds when all hypotheses shattering a set of certain dimension (whose superset has not been shattered) are non overlapping with other hypotheses shattering similar sets.

Informally let' see with an example what this means. Let's say $\mathcal H = \{h_1, h_2,h_3, h_4, h_5, h_6\}$ and all have unique behaviours on $A$ which implies $|\mathcal H_A| = |\mathcal H|$. Now if $\{h_1, h_2,h_3, h_4\}$ shatters a subset $B_1$ of size $2$ then it also shatters all subsets of $B_1$ and number of subsets including $B_1$ itself of $B_1$ is $2^{|B_1|} = 4$ (in this case). Similarly, $\{h_5 h_6\}$ must shatter another subset $B_2$ with size $=1$ and using similar argument as before we get subsets $=2$. Which shows $2+4=6$ (i.e $B_1$ and $B_2$ and all its subsets form the RHS of the equation whose total number is $6$).

NOTE: This will also hold true for hypothesis having same behaviour on subset $A$ i.e if another hypothesis $h_7$ exist which give same output on $A$ as $h_6$. There will be no effect since the former derivation is based on $\mathcal H_A$ and not $\mathcal H$

The case during which $|\mathcal H_A| < |\{B \subseteq A: \mathcal{H} \text { shatters } B\} |$ , is when we have overlapping hypothesis i.e if certain hypothesis exists which helps in shattering $2$ different subsets none of which is subset of each other. Which means now if  $\{h_1, h_2,h_3, h_4\}$ shatters some $B_1$ and $\{h_4, h_5\}$ with the overlap of $h_4$ shatters another hypothesis then this is the case $<$ will hold.

Now coming to the second part of the equation, we can see equality will hold when the hypothesis shatters all possible subsets of size $\leq d$. If we define a hypothesis as a set of points where the values is $1$ and $0$ otherwise, then we can verify this. Clearly, if $\mathcal H$ shatters all sets of size $\leq d$ then we can pick any set of size $i \leq d$, and total number of such sets will be $\sum_{i=0}^d \begin{array}{c}{|A|} \\ {i}\end{array}$. In all other cases $<$ will hold.

NOTE: A hypothesis class having VC dimension $d$ does not necessarily shatter all sets of size less $\leq d$ and hence the inequality. Check tis answer.

You can now play with the different scenarios to get $4$ cases as required, and as can  be clearly seen the assumptions for equality in both the part of the equations are independent of each other. One deals with overlapping behaviour, while other deals with shattering of subsets $\leq d$. But note only under a certain condition will the equality hold on both sides, namely the hypothesis class shatters all possible subsets of size $d$ non overlappingly.

Sidenote: The first part of the equation can probably be proven by formal number theory, but I am not sure how.


Is there a mathematical proof that shows that certain parameters work "better" than others for a certain task? The machine learning community often only provides empirical results, but I am also interested in theoretical results and proofs. Specifically, is there a mathematical proof that shows that certain parameters work "better" than others for a certain task?

There is stuff like the Universal Approximation Theorem.

There are also investigations into the loss surface of neural networks.

And classics like this explanation of the vanishing gradient problem.

But I'm afraid the mathematical theory of neural networks only exists in bits and pieces in many different papers. And many of the most important questions can currently only be answered empirically.

Not really, I mean at it's core machine learning from an application perspective often seeks produce human level results, but there isn't any theorem describing human understanding of reality.

Like proving computer vision works well is essentially like proving you have a correct understanding of human perception.

It becomes somewhat circular, and while there exists proofs for certain qualities of data, none of them are true. I mean think about trying to describing reality, it exists on a lower dimensional manifold but analytically describing it? Don't think so.

Even proving robustness ends up being somewhat futile since even if you correctly eliminate advasarial examples this doesn't mean you CV application will produce correct results in general, only that the classification is robust(robust and correct are two different things).


How can these 7 AI problem characteristics help me decide on an approach to a problem? If this list1 can be used to classify problems in AI ...

  Decomposable to smaller or easier problems
  Solution steps can be ignored or undone
  Predictable problem universe
  Good solutions are obvious
  Uses internally consistent knowledge base
  Requires lots of knowledge or uses knowledge to constrain solutions
  Requires periodic interaction between human and computer

... is there a generally accepted relationship between placement of a problem along these dimensions and suitable algorithms/approaches to its solution?

References

[1] https://images.slideplayer.com/23/6911262/slides/slide_4.jpg

The List

This list originates from Bruce Maxim, Professor of Engineering, Computer and Information Science at the University of Michigan.  In his lecture Spring 1998 notes for CIS 4791, the following list was called,

  "Good Problems For Artificial Intelligence."

  Decomposable to easier problems
  Solution steps can be ignored or undone
  Predictable Problem Universe
  Good Solutions are obvious
  Internally consistent knowledge base (KB)
  Requires lots of knowledge or uses knowledge to constrain solutions
  Interactive

It has since evolved into this.

  Decomposable to smaller or easier problems
  Solution steps can be ignored or undone
  Predictable problem universe
  Good solutions are obvious
  Uses internally consistent knowledge base
  Requires lots of knowledge or uses knowledge to constrain solutions
  Requires periodic interaction between human and computer

What it is

His list was never intended to be a list of AI problem categories as an initial branch point for solution approaches or a, "heuristic technique designed to speed up the process of finding a satisfactory solution."

Maxim never added this list into any of his academic publications, and there are reasons why.

The list is heterogeneous.  It contains methods, global characteristics, challenges, and conceptual approaches mixed into one list as if they were like elements.  This is not a shortcoming for a list of, "Good problems for AI," but as a formal statement of AI problem characteristics or categories, it lacks the necessary rigor.  Maxim certainly did not represent it as a, "7 AI problem characteristics," list.

  It is certainly not a, "7 AI problem characteristics," list.

Are There Any Category or Characteristics Lists?

There is no good category list for AI problems because if one created one, it would be easy to think of one of the millions of problems that human brains have solved that don't fit into any of the categories or sit on the boundaries of two or more categories.

It is conceivable to develop a problem characteristics list, and it may be inspired by Maxim's Good Problems for AI list.  It is also conceivable to develop an initial approaches list.  Then one might draw arrows from the characteristics in the first list to the best prospects for approaches in the second list.  That would make for a good article for publication if dealt with comprehensively and rigorously.

An Initial High Level Characteristics to Approaches List

Here is a list of questions that an experienced AI architect may ask to elucidate high level system requirements prior to selecting an approaches.

Is the task essentially static in that once it operates it is likely to require no significant adjustments?  If this is the case, then AI may be most useful in the design, fabrication, and configuration of the system (potentially including the training of its parameters).
If not, is the task essentially variable in a way that control theory developed in the early 20th century can adapt to the variance?  If so, then AI may also be similarly useful in procurement.
If not, then the system may possess sufficient nonlinear and temporal complexity that intelligence may be required.  Then the question becomes whether the phenomenon is controllable at all.  If so, then AI techniques must be employed in real time after deployment.

Effective Approach to Architecture

If one frames the design, fabrication, and configuration steps in isolation, the same process can be followed to determine what role AI might play, and this can be done recursively as one decomposes the overall productization of ideas down to things like the design of an A-to-D converter, or the convolution kernel size to use in a particular stage of computer vision.

As with other control system design, with AI, determine your available inputs and your desired output and apply basic engineering concepts.  Thinking that engineering discipline has changed because of expert systems or artificial nets is a mistake, at least for now.

Nothing has significantly changed in control system engineering because AI and control system engineering share a common origin.  We just have additional components from which we can select and additional theory to employ in design, construction, and quality control.

Rank, Dimensionality, and Topology

Regarding the rank and dimensions of signals, tensors, and messages within an AI systems, Cartesian dimensionality is not always the correct concept to characterize the discrete qualities of internals as we approach simulations of various mental qualities of the human brain.  Topology is often the key area of mathematics that most correctly models the kinds of variety we see in human intelligence we wish to develop artificially in systems.

More interestingly, topology may be the key to developing new types of intelligence for which neither computers nor human brains are well equipt.

References

http://groups.umd.umich.edu/cis/course.des/cis479/lectures/htm.zip

The 7 AI problem characteristics is a heuristic technique designed to speed up the process of finding a satisfactory solution to problems in artificial intelligence.

In computer science, artificial intelligence and mathematical optimization, a heuristic is a technique designed for solving a problem more quickly, or for finding an approximate solution when you have failed to find an exact solution using classic methods.

The 7 AI problem technique ranks alternative steps based on available information to help one decide on the most appropriate approach to follow in solving problems i.e. missionaries and cannibals, Tower of Hanoi, Traveling salesman e.t.c.

Regarding whether there is a generally accepted relationship between the placement of a problem and suitable algorithms. The answer is that indeed there is a generally accepted relationship.
For example imagine trying to solve a game of chess and a game of sudoku.

If a step is wrong in sudoku, we can backtrack and attempt a different approach. However if we are playing a game of chess and realize a mistake after a couple of moves. We cannot simply ignore the mistake and backtrack.(2nd Characteristic)

If the problem universe is predictable,  we can make a plan to generate a sequence of operations that is guaranteed to lead to a solution. However in the case of problems with uncertain outcomes, we have to follow a process of plan revision as the plan is carried out while providing the necessary feedback. (3rd Characteristic)

Below is an example of the 7 AI problem characteristics being applied to solve a water jug problem.

Image source
https://gtuengineeringmaterial.blogspot.com/2013/05/discuss-ai-problems-with-seven-problem_1818.html


Is there an LSTM-based unsupervised learning algorithm to label a dataset of curves? I have a big amount of light curves (image below).

I am trying to label the points as signal or background (the signal appears usually periodically, several times, for a given light curve).

More precisely, I want to identify the downward spikes (class label = 1) from the background (class label = 0).

However, the data is not labeled. I tried labeling it by hand, and using a bi-directional LSTM succeeds in labeling the data points properly. However, there are thousands of light curves and labeling all of them would take very long.

Is there any good unsupervised approach to do this (unsupervised LSTM maybe, but any other method that might work on time series would do just fine)?

You can try to work with Gated Recurrent Units or GRU. This will solve your problem of with too much latency time that LSTM required. LSTM also doesn't give preference on newer data too. For more information, you can follow more on this great article

GRU Cells


Why isn't conditional probability sufficient to describe causality? I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships.

How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?

Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:

$$
P(A|B) = \frac{P(B|A)P(A)}{P(B)}
$$

So if you have $P(B|A)$, $P(A)$, and $P(B)$, we can determine $P(A|B)$, and similarly you can determine $P(B|A)$ from $P(A|B)$, $P(B)$ and $P(A)$. Just by looking at $P(B|A)$ and $P(A|B)$, it is therefore impossible to tell what the causal direction is (if any).

In fact, probabilistic inference usually works the other way round: When there is a known causal relation, say from diseases $A$ to symptoms $B$, we usually have $P(B|A)$, and are interested in the diagnostic reasoning task of determining $P(A|B)$ from that. (The only other thing we need for that is the prior probability $P(A)$ since $P(B)$ is just a normalization factor.)

Why isn't conditional probability sufficient to describe causality?

Suppose that, when the barometric pressure, in a certain region, drops below a certain level, two things happen

the height of the column of mercury in your barometer drops below a certain level
a storm occurs

We may be tempted to model these relationships with the following graphical model, where each directed edge represents a causal relationship, so, for example, the drop in barometric pressure causes the storm.

However, this graphical (and causal) model is likely wrong (and unintuitive), given that the drop in barometric pressure is likely only correlated with the storm, so it is not the cause of the storm.

How can we see that the drop in barometric pressure is or not the cause of the storm?

We can compare the probabilities $P(A \mid \text{do}(B))$ and $P(A \mid B)$, where $A$ is the event "a storm occurs" and $B$ is the event "drop in barometric pressure". What does $\text{do}(B)$ mean? It means that we force the event $B$ to occur, that is, we force the drop in barometric pressure to occur. Intuitively, what is then the difference between $P(A \mid \text{do}(B))$ and $P(A \mid B)$? In the case of $P(A \mid \text{do}(B))$, we force the event $B$ to always occur. In the case of $P(A \mid B)$, we only and passively look at the cases of event $A$ when event $B$ occurs (without thus forcing $B$ to occur). We now know the difference between $P(A \mid \text{do}(B))$ and $P(A \mid B)$. However, how does this help us to understand that $B$ is not the cause of $A$? If $B$ was a cause of $A$, then, if we forced $B$ always to occur, then the probability of $A$ should also change accordingly. However, imagine that we are (magically) able to drop the barometric pressure, if the probability of $A$ does not change accordingly (in this case, if it does not increase), then the storm is not an effect of the drop in barometric pressure.

To conclude, Judea Pearl would say that do-operators (or interventions) are required to analyze causal relationships.

The article Probabilistic Causation by Stanford Encyclopedia of Philosophy gives a good overview of the (probabilistic) causation (or causality) field. In particular, have a look at section 3, which describes causal modeling (according to Pearl). Causal modeling and inference actually involve several nontrivial concepts that require some time to get familiar with, such as interventions (or do-operations), several basic causal relationships (such as forks, chains and colliders), d-separation or Bayesian networks.


Make 9 AIs to replace Supreme Court justices Since the supreme court is always political, why not program 9 AI robots that use different methods to determine whether a law is constitutional and the outcome of cases. How would engineers go about building this? Would it work?

Politics are driven by a combination of things such as culture, ideology, religion, ethics, morality, desire for money, fame, and power, etc.  Although God has one truth, most people don't know this truth, a few do, and the rest either disagree about it or just don't believe in Him. The Constitution was written by man.  It does not tell us what to do in every situation. It is open to interpretation.  A person's interpretation of the Constitution is driven by his morality, culture, and all the things I mentioned above. AI systems are designed by people so they are driven by their biases.


What's the role of bounding boxes in object detection? I'm quite new to the field of computer vision and was wondering what are the purposes of having the boundary boxes in object detection.
Obviously, it shows where the detected object is, and using a classifier can only classify one object per image, but my question is that

If I don't need to know 'where' an object is (or objects are) and just interested in the existence of them and how many there are, is it possible to just get rid of the boundary boxes?

If not, how does bounding boxes help detect objects? From what I have figured is that a network (if using neural network architectures) predicts the coordinates of the bounding boxes if there is something in the feature map. Doesn't this mean that the detector already knows where the object is (at least briefly)? So, continuing from question 1, if I'm not interested in the exact location, would training for bounding boxes be irrelevant?

Finally, in architectures like YOLO, it seems that they predict the probability of each class on each grid (e.g. 7 x 7 for YOLO v1). What would be the purpose of bounding boxes in this architecture other than that it shows exactly where the object is? Obviously, the class has already been predicted, so I'm guessing that it doesn't help classify better.

A bounding box is a rectangle superimposed over an image within which all important features of a particular object is expected to reside. It's purpose is to reduce the range of search for those object features and thereby conserve computing resources: Allocation of memory, processors, cores, processing time, some other resource, or a combination of them. For instance, when a convolution kernel is used, the bounding box can significantly limit the range of the travel for the kernel in relation to the input frame.

When an object is in the forefront of a scene and a surface of that object is faced front with respect to the camera, edge detection leads directly to that surface's outlines, which lead to object extent in the optical focal plane. When edges of object surfaces are partly obscured, the potential visual recognition value of modelling the object, depth of field, stereoscopy, or extrapolation of spin and trajectory increases to make up for the obscurity.

  A classifier can only classify one object per image

A collection of objects is an object, and the objects in the collection or statistics about them can be characterized mathematically as attributes of the collection object. A classifier dealing with such a case can produce a multidimensional classification of that collection object, the dimensions of which can correspond to the objects in the collection. Because of that case, the statement is false.

  1) If I don't need to know 'where' an object is (or objects are) and just interested in the existence of them and how many there are, is it possible to just get rid of the boundary boxes?

If you have sufficient resources or patience to process portions of the frame that don't contain the objects, yes.

Questions (2) and (3) are already addressed above, but let's look at them in that context.

  2.a) If not, how does bounding boxes help detect objects?

It helps by fulfilling its purpose, to reduce the range of the search. If by thrifty method a bounding shape of any type can be created, then the narrowing of focus can be used to reduce the computing burden on the less thrifty method by eliminating pixels that are not necessary to peruse with more resource-consuming-per-pixel methods. These less thrifty methods may be necessary to recognize surfaces, motion, and obscured edges and reflections so that the detection of object trajectory can be obtained with reliability.

That these thrifty mechanisms to find the region of focus and these less thrifty mechanisms to use that information and then determine activity at higher levels of abstraction are artificial networks of this type or that or use algorithms of this type or that is not relevant yet. First understand the need to reduce computing cost in AI, which is a pervasive concept for anything more complex than tic-tac-toe, and then consider how bounding boxes help the AI engineer and the stakeholders of the engineering project to procure technology that is viable in the market.

  2.b) From what I have figured is that a network (if using neural network architectures) predicts the coordinates of the bounding boxes if there is something in the feature map. Doesn't this mean that the detector already knows where the object is (at least briefly)?

  2.c) So continuing from question 1, if I'm not interested in the exact location, would training for bounding boxes be irrelevant?

Cognition is something AI seeks to simulate, and many hope to have robots like in the movies that can help out and be invaluable friends, like TARS in the Nolan brothers 2014 film Interstellar. We're not there. The network knows nothing. It can train a complex connection between an input signal through a series of attenuation matrices and activation functions to produce an output signal statistically consistent with its loss function, value function, or some other criteria.

The inner layers of an artificial net may, if not directed to do so, produce something equivalent to a bounding region only if velocity of convergence is present as a factor in its loss or value function. Otherwise there is nothing in the Jacobian leading convergence to reduce its own time to completion. Therefore, the process may complete, but not as well as if cognition steps in and decides that the bounding region will be found first and then used to reduce the total burden of mechanical (arithmetic) operations to find the desired output signal as a function of input signal.

  3) Finally, in architectures like YOLO, it seems that they predict the probability of each class on each grid (e.g. 7 x 7 for YOLO v1). What would be the purpose of bounding boxes in this architecture other than that it shows exactly where the object is? Obviously, the class has already been predicted so I'm guessing that it doesn't help classify better.

Reading the section in A Real-Time Chinese Traffic Sign Detection Algorithm Based on Modified YOLOv2, J Zhang, M Huang, X Jin, X Li, 2017, may help further comprehension of these principles and their almost universal role in AI, especially the text around their statement, "The Network Architecture of YOLO v2 YOLO employs a single neural network to predict bounding boxes and class probabilities directly from full images in one inference. It divides the input image into S × S grids." This way you can see the use of these principles in the achievement of specific research goals.

Other such applications can be found by simply reading the article full text available on the right side of an academic search for yolo algorithm and using ctrl-f to find the word bound.

In principle, you could train the model to output a sigmoid map of coarse object positions (0 -> no object, 1 -> an object center is located here). The map could be subjected to non-maximum suppression and such model could be trained end-to-end. That would be possible, if that's what you are asking.


Can stochastic gradient descent be properly used in any sample based learning algorithm in Reinforcement Learning? Assuming we use an MSE cost function of the form
$$ \sum_s\mu(s)(V_{\pi}(S_t)-\hat{V}(S_t,\theta_t))^2 =  E_{\mu(s)}[(V_{\pi}(S_t)-\hat{V}(S_t,\theta_t))^2])$$
The Stochastic Gradient Descent is used to approximate the true update algorithm, which looks like this
$$\theta_{t+1} = \theta_{t} - \frac{\eta_t}{2}\nabla_{\theta}(E_{\mu(s)}[(V_{\pi}(S_t)-\hat{V}(S_t,\theta_t))^2])$$
to this
$$\theta_{t+1} = \theta_{t} - \frac{\eta_t}{2}\nabla_{\theta}(U_t-\hat{V}(S_t,\theta_t))^2$$
where, for simplicity, $U_t$ represents an unbiased estimate of the true value function $V_{\pi}(s_t)$. This expression is the source of many learning algorithms used in reinforcement learning.
One of the conditions for SGD requires that samples used for updating the parameters must be I.I.D according to the distribution $\mu(s)$. However, in both on-policy and off-policy learning methods, updates at each time-step are based on trajectories generated. Since, along a trajectory, the state $s_{t+1}$ depends on the state at ${s_t}$, this means that the sample used to update $\theta_t$ and $\theta_{t+1}$ are not independent. Many, if not all, sample-based learning algorithms used in RL rely on using SGD, such as the Gradient Monte Carlo Algorithm but I've not really seen anywhere that mentions these algorithms have the "issue" that I mention so I feel like I'm missing something. More formally,

My Question: Does the fact that parameter updates are not I.I.D mean we can't really use stochastic gradient descent AT ALL in learning algorithms, and, if so, why then do these algorithms "work"?

As far as I know, this question applies equally to all forms of parameterised function approximation that are used with learning algorithms (tabular functions*, linear functions and non-linear functions). But, if anyone knows a special reason as to whether these cases should be treated separately could they make clear why
*I understand that when learning algorithms with tabular functions, there exists theory beyond SGD that ensures convergence, however, I'm not entirely sure what this theory is and whether if this makes them exempt, so if anyone knows whether or not it does make them exempt could they also make this clear!

Edit:
It has been highlighted in the comments that replay buffers have been used to resolve the issue of correlated sampling in cases such as DQN and variants of it. This implies correlated sampling is an issue in these cases. Aside from this, I've not heard of replay buffers being used elsewhere (correct me if I'm wrong), so why are replay buffers needed with this off-policy NN approach but not in other learning algorithms given that they all suffer from the issue of correlated sampling.

First I will address the issue of Tabular methods. These do not use SGD at all. Although the updates are very similar to an SGD update there is no gradient here and so we are not using SGD. Many Tabular methods are proven to converge, for instance the paper by Chris Watkins titled "Q-Learning" introduces and proves that Q-learning converges. Also you include tabular methods as being parameterised function approximators. This is not true. Tabular methods maintain an estimate of the value function in a look-up table for each state-action pair and there is no function approximator being used.
Now for non-tabular methods, i.e. Deep Reinforcement Learning. Here we are using SGD only to optimise the parameters of the networks (assuming of course that we are using NN's as our function approximators, but this is a fair assumption if you read the literature). This is why off-policy methods are typically preferred because they can use a replay buffer which allows the use of data from any past trajectory. When using a replay buffer we sample random past experiences which de-correlates the data and allows the i.i.d. assumption to hold when using SGD.
If we were to use an on-policy algorithm, in theory SGD may not converge to any local optima because we are violating the i.i.d. assumption. However, all this means is that we are not guaranteed to converge. For instance, I have run an experiment using REINFORCE (an on-policy learning algorithm) using NN's as function approximators and they were able to obtain an optimal policy. However, this was for a very simple environment using modestly sized networks, so this is likely why they were able to be trained using non-i.i.d. data. A more in depth question/answer as to why NN's require i.i.d. data can be found here.
To address your edit, Replay Buffers are not required but if we can use them, then why would we not? It helps to maintain the i.i.d. assumption and so it helps us obtain a local optima. If we did not use them then there would not be the guarantee that it would converge. As an aside, Replay Buffers are typically used because they make the algorithm have a greater sample efficiency - this means we can obtain an optimal policy using much less data.
If you are wondering why don't on-policy methods use a replay buffer, the answer is because they are on-policy. The actions used in updates must be taken according to our current policy that we are learning the value functions for (or are optimising the policy of in Policy Gradient Methods). This is not the case in off-policy algorithms - e.g. in Q-learning we are learning the value functions of the greedy policy but we follow a different policy that allows for exploration.

As David Ireland has mentioned in his answer, despite correlated sampling, learning algorithms can still converge to the optimal policy/value function.
The reason learning algorithms still may produce the correct results despite the use of correlated sampling is to do with another property, that does not rely on i.i.d sampling, that is, the average regret of SGD tends to zero at the limit. A couple of sources I found that highlight this are

Page 26 of the slides available from the computer science department at the university of British Columbia Machine Learning and Data mining

The introduction of a paper discussing non-iid stochastic optimisation - A. Agarwal and John C. Duchi

Minimising average regret implies that the SGD will, at the limit, produce the best set of parameters that perform well on the encountered samples however this is does not indicate whether the best set of parameters actually performs well.
Extra
This is in fact a general issue that extends beyond the reinforcement learning algorithms. Programs that implement SGD don't typically sample IID! as mentioned in an MIT lecture on stochastic gradient descent at around 40 minutes the lecturer discusses it's more efficient to sample without replacement from the training set making it non IID for which there is not much theory about.

caveat
I'm not 100% sure what the implication of minimising regret means but given regret is of the form
$$regret(\theta_1,\ldots,\theta_t) = \sum_{t=0}^T\left(L(v_\pi(s_t),\hat{v}(s_t,\theta_i) -L(v_\pi(s_t),\hat{v}(s_t,\theta_*)\right)$$
Then I understand the model that minimises regret at time T as the one that has the minimum loss for all training samples encountered up to time T . But what I'm not sure about is how this relates to minimising the cost function, it probably requires another question!


Could a neural network detect primes? I am not looking for an efficient way to find primes (which of course is a solved problem). This is more of a "what if" question.
So, in theory, could you train a neural network to predict whether or not a given number $n$ is composite or prime? How would such a network be laid out?

Early success on prime number testing via artificial networks is presented in A Compositional Neural-network Solution to Prime-number Testing, László Egri, Thomas R. Shultz, 2006.
The knowledge-based cascade-correlation (KBCC) network approach showed the most promise, although the practicality of this approach is eclipsed by other prime detection algorithms that usually begin by checking the least significant bit, immediately reducing the search by half, and then searching based other theorems and heuristics up to $floor(\sqrt{x})$.  However the work was continued with Knowledge Based Learning with KBCC, Shultz et. al. 2006
There are actually multiple sub-questions in this question.  First, let's write a more formal version of the question: "Can an artificial network of some type converge during training to a behavior that will accurately test whether the input ranging from $0$ to $2^n-1$, where $n$ is the number of bits in the integer representation, represents a prime number?"

Can it by simply memorizing the primes over the range of integers?
Can it by learning to factor and apply the definition of a prime?
Can it by learning a known algorithm?
Can it by developing a novel algorithm of its own during training?

The direct answer is yes, and it has already been done according to 1. above, but it was done by over-fitting, not learning a prime number detection method.  We know the human brain contains a neural network that can accomplish 2., 3., and 4., so if artificial networks are developed to the degree most think they can be, then the answer is yes for those.  There exists no counter-proof to exclude any of them from the range of possibilities as of this answer's writing.
It is not surprising that work has been done to train artificial networks on prime number testing because of the importance of primes in discrete mathematics, its application to cryptography, and, more specifically, to cryptanalysis.  We can identify the importance of digital network detection of prime numbers in the research and development of intelligent digital security in works like A First Study of the Neural Network Approach in the RSA Cryptosystem, G.c. Meletius et. al., 2002.  The tie of cryptography to the security of our respective nations is also the reason why not all of the current research in this area will be public.  Those of us that may have the clearance and exposure can only speak of what is not classified.
On the civilian end, ongoing work in what is called novelty detection is an important direction of research.  Those like Markos Markou and Sameer Singh are approaching novelty detection from the signal processing side, and it is obvious to those that understand that artificial networks are essentially digital signal processors that have multi-point self tuning capabilities can see how their work applies directly to this question.  Markou and Singh write, "There are a multitude of applications where novelty detection is extremely important including signal processing, computer vision, pattern recognition, data mining, and robotics."
On the cognitive mathematics side, the development of a mathematics of surprise, such as Learning with Surprise: Theory and Applications (thesis), Mohammadjavad Faraji, 2016 may further what Ergi and Shultz began.

yes it is feasible, but consider that integer factorization problem is an NP-something problem and BQP problem.

because of this, it is impossible that a neural network purely based on classical computing finds prime number with 100% accuracy, unless P=NP.


What activation function does the human brain use? Does the human brain use a specific activation function? I've tried doing some research, and as it's a treshold for whether the signal is sent through a neuron or not, it sounds a lot like ReLU. However, I can't find a single article confirming this. Or is it more like a step function (it sends 1 if it's above the treshold, instead of the input value).

The thing you were reading about is known as the action potential. It is a mechanism that governs how information flows within a neuron.

It works like this: Neurons have an electrical potential, which is a voltage difference inside and outside the cell. They also have a default resting potential, and an activation potential. The neuron tends to move towards the resting potential if it is left alone, but incoming electric activations from dendrites can shift its electric potential.

If the neuron reaches a certain threshold in electric potential (the activation potential), the entire neuron and its connecting axons goes through a chain reaction of ionic exchange inside/outside the cell that results in a "wave of propagation" through the axon.

TL;DR: Once a neuron reaches a certain activation potential, it electrically discharges. But if the electric potential of the neuron doesn't reach that value then the neuron does not activate.

  Does the human brain use a specific activation function?

IIRC neurons in different parts of the brain behave a bit differently, and the way this question is phrased sounds as if you are asking if there is a specific implementation of neuronal activation (as opposed to us modelling it).

But in general behave relatively similar to each other (Neurons communicate with each other via neurochemicals, information propagates inside a neuron via a mechanism known as the action potential...) But the details and the differences they cause could be significant.

There are various biological neuron models, but the Hodgkin-Huxley Model is the most notable.

Also note that a general description of neurons don't give you a general description of neuronal dynamics a la cognition (understanding a tree doesn't give you complete understanding of a forest)

But, the method of which information propagates inside a neuron is in general quite well understood as sodium / potassium ionic exchange.

  It (activation potential) sounds a lot like ReLU...

It's only like ReLU in the sense that they require a threshold before anything happens. But ReLU can have variable output while neurons are all-or-nothing.

Also ReLU (and other activation functions in general) are differentiable with respect to input space. This is very important for backprop.

This is a ReLU function, with the X-axis being input value and Y-axis being output value.

And this is the action potential with the X-axis being time, and Y being output value.

My interpretation of the question was 'what activation function in an artificial neural network (ANN) is closest to that found in the brain?'

Whilst I agree with the selected answer above, that a single neuron outputs a dirac, if you think of a neuron in an ANN as modelling the output firing rate, rather than the current output, then I believe ReLU might be closest?

http://jackterwilliger.com/biological-neural-networks-part-i-spiking-neurons/