Could artificial intelligence cause problems for humanity after figuring out human behavior? This BBC article suggests that intelligent algorithms, like those that select news stories and advertisements for display, could control our experience of the world and manipulate us.

Will Artificial Intelligence someday become a problem to humanity after learning human behaviors and characteristics?

Will Artificial Intelligence some day become a problem to humanity
  after learning human behaviors and characteristics?

It can be answered in both ways, I think.

Yes, they may become a problem.

With the increasing integration of loads of apps and smart devices in our life, almost everything defining an individual human being is digitalised. For instance, our fingerprints, voice, facial image etc. Apart from these data, we use those apps and devices to track our health (heart rate, calorie intake etc), to plan our schedules, and most importantly to communicate. If some sort of AI engine is integrated into a chat application, for instance, it can learn our typing patterns, conversation style, and hundreds of other unforeseen parameters. Imagine what can be learned about a person if such AI is coded inside every device and every app in your day-to-day use.

We use smart devices and apps to harness their functionalities and features which ease our way of life, and we give them, unintentionally, our identity and sometimes, even our personality. For them, these are the parameters that can be input to some machine-learning algorithm and predict what we will do the next day, or what will happen to us the next day.

This sounds like a major problem, especially when these technologies are indispensable.

No, they may not become a problem.

Humans are really complex creatures and possess the most advanced intelligence technology called the brain. I think the brain can be thought of as a technology in this context. There are several tissues inside the brain that can learn to do certain things themselves. It was proven in a research that some tissues have the capability to perform the functions of other tissues using neuro-rewiring techniques. Imagine that the same tissue that has helped you see until now can be made to help you hear instead. Now imagine mimicking such a technology.

While it is not impossible, for an AI to achieve the brilliance of a human brain is a topic of ongoing research. To train a machine for the purpose, we would have to feed it with gazillion behaviors and characteristics, which it may not be able to handle! After learning some behavior and characteristics, the AI would be said to be smart, but it would still predate us.

So, they may not become a problem at all because of our brain.

The algorithms (like recommender systems) used by Amazon and Facebook influence us or even manipulate us. But, that manipulation is either very obvious (like viewing promoted products) or is in company's best interests (like viewing a certain news piece). It may be even possible that several external parameters are used by these systems to improve your experience. For instance, Google ads show us what we were looking for on an online store when we visit any random website. In most cases, what you see is a result of what you were looking for before. If any attempt to influence does happen, we may learn to avoid it through careful observation or even experience.

An optimistic view is no just like any other inventions we had in the past they improved the quality of life.
A pessimistic view maybe the algorithm may control all the computer systems globally and enslave us.
 A realistic view is that AI will forever be there to serve us, for as the pessimistic view may actually makes sense in the fictional world ... the truth is its impossible because the compute capability required is extravagant.


Recognition of lines in a chalkboard I'm trying to develop a real-time application that, from the sequence of chalkboard images captured by a webcam, recognizes the lines being draw on it.

It must be able of recognize the lines from the chalkboard background, filter the presence in the image of the teacher, and translate these lines to some representation, something as a list of basic events like "start of line at xxx,xxx", "continue line at xxx,xxx", ...

After several days looking for references and bibliography, none is found. The most similar are the character recognition applications, in particular when they have a stroke recognition stage.

Any hint ?

Input will be a sequence as this one,  this one or this one (just without the presence of the students). I've expect the teacher not hidding his hand. We could imagine a start with an empty chalkboard.

Thanks.

Note: I am looking for more than an answer which says only something similar to "you can use a deep learning training it with two classes", without details or references.

I will assume that the camera is stable (no change in position, zoom or other settings during the video recording), otherwise the task becomes markedly more complicated.

Let's say that your dataset is an array of rasters (images in array format). You mention that you want to detect events "start line" and "end line".

One way of doing this would be to compute an approximate time derivative of your image series. For instance, take the image raster at index idx and the one right after, at index idx+1 (captured at instants $t$ and $t + \Delta t$, where $\Delta t$ is the sampling interval).

At coordinates $(i,j)$, this derivative could look something like: timeDerivative = (images[idx][j][i] - images[idx+1][j][i])/DeltaT. This is a crude estimate, and there are better ways of computing an approximate discrete derivative, but you get the idea.

Following this, you could declare a state of the recording: drawing line or not drawing line. The states, we assume are always alternating, as a teacher has to take his hand off the blackboard to draw a new line.

When a derivative with large values is detected (a region in the image goes from being black to white suddenly) and the state is "not drawing", the event "start line" is recorded and the state switches to "drawing". While a derivative with large values continues to be detected as time goes on in the vicinity of the previous spot with a large derivative, nothing changes. Once this is no longer true, the state changes to "not drawing" and the event "stop line" is recorded at the last location with a large derivative.

This is the main idea, which can be improved with:

Defining the area of the blackboard, either by hand or automatically
Thresholding the images to better isolate the chalk trace from the blackboard
Using a tracker such as a Kalman filter to know where to look next for the chalk

Maybe Just use simple Convnets (Pre-trained perhaps) and train it on the images of the teacher on the blackboard. You could use a GAN to remove the teacher and complete the rest of the image (https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwj9lKKSh43uAhWbfH0KHeaDC1UQFjAAegQIAxAC&url=http%3A%2F%2Fstanford.edu%2Fclass%2Fee367%2FWinter2018%2Ffu_guan_yang_ee367_win18_report.pdf&usg=AOvVaw2tG3bStRlys_NPLX9-XPep) But that would be too troublesome.
The best way would be to take real-time video chunks, use a Convolutional Network to detect the shapes you want, and return bounding boxes for the appropriate shapes (for the location purposes) and their 'classification' - whether a straight line, curved line, or some other user defined shape. You could also choose to use some other YOLO (You Only Look Once) technique. You can check out with: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjAJegQIARAC&url=https%3A%2F%2Ftowardsdatascience.com%2Fobject-detection-using-deep-learning-approaches-an-end-to-end-theoretical-perspective-4ca27eee8a9a&usg=AOvVaw2TkW094_O5Q7-mcVMJ5SEN ;
You also won't have to deal with the pesky teacher with the above method (assuming he doesn't stand in one place and constantly block a part of the BB). These methods are near SOTA and would be extremely effective than conventional algos. Not to mention that using Keras is a piece of cake with a giant community and endless resources to help you in case you get stuck on some problem. Since it is easy to use, you can setup a prototype in almost no time.
Beginner's guide and Introduction-> https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjALegQIGhAC&url=https%3A%2F%2Fmachinelearningmastery.com%2Fobject-recognition-with-deep-learning%2F&usg=AOvVaw3M-b1gYnbTdzzwPahTTWWR ;
A research paper from ArXiv--> https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiei4C3iI3uAhXXfX0KHQXxAVAQFjAPegQIHhAC&url=https%3A%2F%2Farxiv.org%2Fpdf%2F1807.05511&usg=AOvVaw33CWoOX4LlrA3T_f75zVZu
Training with YOLO: https://machinelearningmastery.com/how-to-perform-object-detection-with-yolov3-in-keras/


Can an AI distinguish between good and bad according to people living in a restricted geographical area? Would people go far with Artificial Intelligence and machine learning to the point where machines could learn during a long period of time to distinguish what's 'good' from 'bad' according to people living in a restricted geographical area, and then the machines take control and turn what was learnt into a set of 'rules' and 'laws' (think of it as an effective machine of 'politics') that match the majority of the people's view of issues.

That should be accepted by everyone, since a contract set at the beginning says: "Everyone is ok".

Even if you could do that (which I believe is a long way off), what would be the point?

If I understand you correctly, you want an AI system to learn through observation of human beings what their 'rules of interaction' are. It sees a person killing someone else, and then that person is punished by the community, so the AI learns that killing people is not right. However, that is already codified in laws... So what it would pick up are social behaviours, which hopefully be things like "be nice to other people", "don't do anybody any harm", "be honest and truthful", etc.

The first question then is how an AI could assess events[*] as to whether they are 'good' or 'bad'. If everybody lies and steals, that would be learnt as normal behaviour, and the AI would not be able to pick up that most people would see this as 'bad'[**]. Causal relations are also hard to grasp. Somebody steals something. Then later someone else buys him a drink. So stealing stuff means other people will buy you drinks? These are really hard problems to solve. You need to know about people's motivations, and the multitude of 'threads' of interaction happening at the same time, even in a very limited area.

So, recognising events and causal links, plus a moral evaluation of them is pretty difficult. I don't think we will get there anytime soon. Unsupervised learning of behavioural is also pretty difficult, as you only have unlabelled observations and no real criteria to classify them. Plus, many actions are morally ambiguous. Killing people is generally seen as bad. What about the officers who tried to assassinate Hitler in 1944? Life is complex, and our artificial models are not anywhere near that.

So even if you were able to do all that, what would you end up with? An AI system that has picked up a lot of unwritten rules about human behaviour, and then postulates that as laws? So everyone has to behave the same way? I just don't see the point of that, even as a thought experiment.

[*] leaving aside here the question how you determine what an 'event' is in the first place
[**] Please note that even if people steal things, they steal can view theft as a bad thing.

The most interesting scientific field in terms of relevance to this question is probably "affective computing".

There are several problems with the model that you suggested. It is questionable if AGI should learn in the same way humans learn. In addition, there are several ethical problems surrounding this question, perhaps even metaethics, because the question peeks beyond human ethics.

A hardcoded ethics protocol might be possible to implement, similar to Isaac Asimov's "Three Laws of Robotics". The possibility of hacking is an issue that one would have to think proactively about - especially in this case.

If humanoid AGIs or robots are very similar to humans, they should probably be treated that way. For example, ethiologist Frans de Waal has studied empathy and social behavior in monkeys and suggested that we are very similar in behavior, so we are probably similar in terms of feelings (if it quacks like a duck and so forth). Perhaps we need an ethiology for androids too?


When should I use simulated annealing as opposed to a genetic algorithm? What kind of problems is simulated annealing better suited for compared to genetic algorithms?

From my experience, genetic algorithms seem to perform better than simulated annealing for most problems.

Simulated Annealing vs genetic algorithm?

Simulated annealing is a materials science analogy and involves the introduction of noise to avoid search failure due to local minima. See images below. To improve the odds of finding the global minimum rather than a sub-optimal local one, a stochastic element is introduced by simulating Brownian (thermal) motion. Variants of simulated annealing include injection of random numbers with various distributions and the averaging effect of mini-batching (dividing a batch into segments and performing network parameter adjustment after each).

Genetic algorithms are search methods based on principles of mutation, meiosis, symbiosis, test, elimination of inadequacy, and recursion. The advantages of such approaches is the simulation of sexual reproduction, where the possibility of dominant genetic features from two individuals producing a child individual containing the best of both exists. In a population of children, the probability of such a hybrid emerging is higher. Over several generations, even higher than that.

  What kind of problems does simulated annealing perform better than genetic algorithms if any?

This question can only be answered well if comparing original, pure versions of both, not the variants that have developed since their introduction.

Simulated annealing or other stochastic gradient descent methods usually work better with continuous function approximation requiring high accuracy, since pure genetic algorithms can only select one of two genes at any given position.

  From my experience, genetic algorithm seems to perform better than simulated annealing for most problems.

Those performance results would be of value to others and should be published as a paper, presented as an open source project, or published creative commons in the appropriate AI venue. At the very minimum, the results could be placed in the above question or an answer to it.

References

[1] Optimization by Simulated Annealing, Science S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, Science New Series, Vol. 220, No. 4598, May 1983, pp. 671-680

[2] N. Metropolis, A. Rosenbluth, M. Rosenbluth., A. Teller. E. Teller, J. Chem. Phys. 21. 1087 110511

Optimised simulated annealing for Ising spin glasses, 2015, S.V. Isakov et. al.
A parallel simulated annealing method for the vehicle routing problem with simultaneous pickup–delivery and time windows, 2014, Chao Wang et. al.
Feature-based tuning of simulated annealing applied to the curriculum-based course timetabling problem, 2015, Ruggero Bellio et. al.

Images from Other Answers

If digital values are mere estimates, why not return to analog for AI?
Method to check goodness of combinatorial optimization algorithm implementation, plotted from https://github.com/facebookresearch/nevergrad/blob/master/nevergrad/functions/corefuncs.py

Simulated annealing algorithms are generally better at solving mazes, because they are less likely to get suck in a local minima because of their probabilistic "mutation" method. See here. Genetic algorithms are better at training neural networks, because of their genetically inspired training algorithm. This makes them more versatile and efficient in more complex situations.


Why does Batch Normalization work? Adding BatchNorm layers improves training time and makes the whole deep
model more stable.  That's an experimental fact that is widely used in machine
learning practice.
My question is - why does it work?
The original (2015) paper motivated introduction of the layers by stating
that these layers help fixing "internal covariate shift". The rough idea is
that large shifts in the distributions of inputs of inner layers makes training
less stable, leading to decrease in the learning rate and slowing down of the
training.  Batch normalization mitigates this problem by standardizing the
inputs of inner layers.
This explanation was harshly criticized by the next (2018) paper -- quoting
the abstact:

... distributional stability of layer inputs has little to do with the success of BatchNorm

They demonstrate that BatchNorm only slightly affects the inner layer inputs
distributions.  More than that -- they tried to inject some non-zero
mean/variance noise into the distributions. And they still got almost the
same performance.
Their conclusion was that the real reason BatchNorm works was that...

Instead BatchNorm  makes the optimization landscape significantly smoother.

Which, to my taste, is slightly tautological to saying that it improves
stability.
I've found two more papers trying to tackle the question: In this paper
the "key benefit" is claimed to be the fact that Batch Normalization biases
residual blocks towards the identity function. And in this paper that it
"avoids rank collapse".
So. Is there any bottom line? Why does BatchNorm work?

To some extend, it get rid of low intensity numerical noise. Condition properties of the optimization problem is always an issue, i suspect BatchNorm alleviate this instability.


Is is not possible to achieve average reward of more than 20-40 with simple Q-Learning I have implemented the simple Q-Learning based solution for AI-gym's Cartpole-v0.
However, despite changing hyper-parameters, and rechecking my code, I cannot get an average reward (N-running reward) of more than 30. My question is, is it not possible to get successful completion of Cartpole without using sophisticated algorithms such as Deep learning etc.?
I am glad to share my code, but I am sure no one would have time to check it.

PS. I know there are many implementations out there, but I have learned from them but I want to implement my own code for learning purpose and do not just want to copy-paste.
PSS (Edit): I have added the code in the answer to this question for reference.

The code to my question is as below, for reference:
import gym
import numpy as np
import matplotlib.pyplot as plt

# Discretize the contiuous space
DISCRETE_POINTS = 50
X_position = np.linspace(-2.4, 2.4, DISCRETE_POINTS)
Velocity = np.linspace(-5, 5, DISCRETE_POINTS)
Angle = np.linspace(-0.7295476, 0.7295476, DISCRETE_POINTS)
Angular_vel = np.linspace(-5,5,DISCRETE_POINTS)

# Fit the instantnous state into any discrete box
def get_state(_state):

    X, X_bar, Angle_, Angle_bar = _state

    X = int(np.digitize(X, X_position))
    X_bar = int(np.digitize(X_bar, Velocity))
    Angle_ = int(np.digitize(Angle_, Angle))
    Angle_bar = int(np.digitize(Angle_bar, Angular_vel))

    return (X, X_bar, Angle_, Angle_bar)

def epsilon_greedy_action(s, epsilon):

    '''   Input argument: state 's' tuple in the form (4,0,1,0)    '''
    # if np.random.uniform() < epsilon:
    if np.random.random() > epsilon:
        a = env.action_space.sample()
    else:
        _,a = find_maxQ_value(s)

    return a

def find_maxQ_value(state):
    '''
    Input argument:
    state: should be a tuple of form (0,0,0,0) or (1,0,0,1) etc.

    Output argument:
    best_value: best q-value of the current state-action pair
    choosen_action: best action corresponding to current state. It depends on the best q-value
    '''

    for act_ in range(env.action_space.n):
        A = [Q[state,0], Q[state, 1]]
        best_value = np.max(A)
        choosen_action = np.argmax(A)

    return best_value, choosen_action

def plotRunningAverage(totalrewards, N, n_avg):

    running_avg = np.empty(N)
    for t in range(N):
        running_avg[t] = np.mean(totalrewards[max(0, t-N):(t+1)])
    return running_avg

if __name__ == '__main__':

    env = gym.make('CartPole-v0');

    EPISODES = 1000;
    no_actions = env.action_space.n

    # Hyper parameters
    alpha = 0.001 # Learning rate
    gamma = 0.99 #Discount Factor
    epsilon = 1 # For Epsilon-Greedy algorithm
    epsilon_decay_factor = 0.99;
    min_epsilon = 0.1;

    states = []
    for i in range(len(X_position)):
        for j in range(len(Velocity)):
            for k in range(len(Angle)):
                for l in range(len(Angular_vel)):
                    states.append((i,j,k,l))

    #Initialize Q-table :
    # 1. We make the Q-table in form of a dictionary
    # 2. We initialize Q-table values as zero in this
    Q = {}
    for s in states:
        for n_a in range(no_actions):
            Q[s, n_a] = 0

    Running_reward = [];
    l_action_cnt = 0;
    r_action_cnt = 0;
    wrong_action =0;

    #Q-Learning agent episodes
    for e in range(EPISODES):

        cn_state = env.reset()
        ds_state = get_state(cn_state)

        done = False
        ep_reward = 0
        ep_len = 0

        while not done:

            action = epsilon_greedy_action(ds_state, epsilon)
            if action == 0:
                l_action_cnt+=1
            elif action == 1:
                r_action_cnt+=1;
            else:
                wrong_action+=1;

            cn_next_state, reward , done , ep_len = env.step(action)
            ep_reward += reward

            ds_next_state = get_state(cn_next_state)

            # Update the Q-table based on the action
            Val_Q_bar, _ = find_maxQ_value(ds_next_state);

            Q[ds_state, action] = (1-alpha)*Q[ds_state, action] + alpha*(reward + gamma*Val_Q_bar)

            ds_state = ds_next_state;

        Running_reward.append(ep_reward)

        if e%100 == 0:
            print('Episode : {}, Episode reward: {}, Epsilon: {}'.format(e, ep_reward, epsilon))

        if epsilon >= min_epsilon:
            epsilon*=epsilon_decay_factor;

plt.plot(Running_reward);
plt.xlabel('episodes')
plt.ylabel('episodic reward')
plt.grid('ON')
running_avg = plotRunningAverage(Running_reward, EPISODES, 50)
plt.plot(running_avg);
plt.legend(['Episodic rewards', '50-Episode moving-average reward'])

print('The ratio of left to right action is : {}'.format(l_action_cnt/r_action_cnt))


What kind of AI technique can I use to play the "Lines" game? I am trying to find a good approach to create a computer player for the game "Lines" from gamious on Android. The concept of the game is pretty straightforward :

Lines is an abstract ‘zen’ game experience where form is just as important as function. Place or remove Dots to initiate a colourful race that fills a drawing. The colour that dominates the race wins.

The game starts with a drawing (that can be described as a set of "blank" lines, with connection to other lines). Dots of different colour are placed somewhat randomly on the lines. The player get a colour assigned. When the game start, paint start flowing from the dots and filling the (at first blank) lines of the drawing. You win if your colour dominates.
The game gives you different tools to win (the game starts when all of them have been used) :

[0 to 2] scissor to cut lines
[0 to 5] additional dot of your own color to place on the drawing
[0 to 4] enemy dots eraser
[0 to 3] additional straight lines to connect different part of the drawing

A quick example: the first image is the initial state of a round. "My" colour is the yellow (1 enemy = brown) and I have 4 tools (2 eraser and 2 lines). The second image shows the game running after I used the tools to put my colour in a winning position (yes, we can do better)

If I try to approach this as a classical optimization problem, things get messy pretty fast :

highly non-linear
high number of dimensions

AI seems to be the right way to go, but I would like your help to get in the right direction: what would be your approach to create an AI to play this game?
To limit the scope of this question, you can consider that I already have a data structure to represent the game initial state, the use of different tools and the game "physics". What I really want to do is finding how to create an AI which can learn how to efficiently use the tools.
Regarding my experience, I took 2 semesters of AI classes during the last year getting my engineering degree and have used non-linear optimization tools for a while: you can go technical, but I am not fully understand it.

Writing a game AI is difficult in general and it's even harder when there's no game to play. Like in your case.

While "Lines" is pretty neat, there's no (playing) opponent and there are no moves. apart from the setup. So what we have is an optimization problem where we need to get more point than the "opponents".

Game playing algorithms like Reinforcement learning or MCTS are IMHO out of questions as there's no game to play. Still, it's an interesting problem, though I can't see how to make any learning AI. Anyway, what you need first is a simulator and an optimizer choosing promising actions.

Some ideas concerning the actions

Some actions are trivial, e.g., the eraser as there are just a few dots to erase. Trying all combination should be good enough for the start.

The scissors can be in theory placed anywhere, but the only interesting places are just next to the oponent piece or a crossing. Additionally, you only need to consider crossings which gets "won" (reached first) by an opponent. Moreover, they only matter if they get won by an opponent having more points than you or blocking you.

The dots to insert should similarly by placed near an oponent or so that you just win a crossing.

The additional lines are worst, as they may be used to let you reach unconnected areas, or reach a crossing faster or simply give you additional points (then you want a long line somewhere).

A model

Most of the game can be seen as a non-directed graph with edges labeled by their length, where the crossing and the initial dots are nodes. The simulator doesn't need to work "pixel by pixel", but may instead jump to the next time, when a crossing gets reached or two "flows" meat each other. The points of a player grow linearly with time, where the slope is given by the number of their extending "flows".

So far, I could imagine modelling it as an Integer Linear Program, but with the additional lines, it's probably no more possible.

I'd probably let the optimization be simulator-guided, i.e., analyse whether I could get closer to winning by reaching a crossing sooner. The details are still unclear....


In the multi-linear regression, how is the value of weight $b_2$ calculated? In multivariate linear regression (linear regression with more than one variable) the model is $yi = b_0 + b_1x_{1i} + b_2x_{2i} + ...$ , and so on. But how is the $b_n$ value calculated iteratively? Can it be calculated non-iteratively? What is the intuition behind using that method to calculate $b_2$?

It is calculated the same way $b_1$ is calculated.

Nearly following your notation, say your multiple linear regression function is

$H(X_i) =  b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$

for data instance $X_i=x_{1,i},...,x_{n, i}$
and weights $b_0,...,b_n$.

And say your error function is $E(X,Y) = \sum_i(H(X_i)-Y_i)^2$

where $X$ is the collection of all data points $X_i, Y_i$.

From your error function $E$, for whatever weights you have (with a gradient based method), calculate the partial derivative
$\partial E /\partial b_i$ and use this to update all of your weights at once in each iteration of your optimization routine.

put your b_1, ... ,b_n coefficients into a vector b

put all your x_{ij} into a matrix X

then all components of b are calculated at the same time with this equation

b = (H(X) - Xb)^T (H(x) - Xb)

But this calculation (estimation) is only consistent (search consistent estimator) when certain assumptions are present (read here https://en.wikipedia.org/wiki/Ordinary_least_squares).


What types of applications qualify as "compound intelligences"? A question about swarm intelligence as a potential method of strong general AI came up recently, and yielded some useful answers and clarifications regarding the nature of swarm intelligence. But it got me thinking about "group intelligence" in general.

Here organism is synonymous with algorithm, so a complex organism is an algorithm made up of component algorithms, based on a set of instructions in the form of a string.

Now consider the Portuguese man o' war, not a single animal, but a colonial organism. In this case, that means a set of animals connected for mutual benefit.

And physalia physalis are pretty smart as a species in that they've been around for a while, I'm not finding them on any endangered lists, and based on their habitat it looks like global warming will be a jackpot for them. And they don't even have brains.

Each component of the physalia has a narrow function, colony organism itself has a more generalized function, which is the set of functions necessary for maintenance and reproduction.

{Man o' War} ⊇ { {pneumatophore}, {gonophores, siphosomal nectophores,vestigial siphosomal nectophores}, {free gastrozooids, tentacled gastrozooids, gonozooids, gonopalpons}, {dactylozooids}, {gonozooids}, {gastrozooids} }

What types of applications qualify as "compound intelligences"? What is the thinking on groups of neural networks comprising generally stronger or simply more generalized intelligence?

I recognize the underlying problem is ultimately complexity and that "strong narrow AI" is, by definition, limited, so I use "generalized" and omit "strong" because human-like and superintelligence are not conditions.  Compound intelligence is defined as a colony of dependent intelligences.*

Utility software is often a form of an expert system that manages a set of functions of varying degrees of complexity. There's currently a great deal of focus on autonomous vehicles, which would seem to require sets of functions.

Links to research papers on this or related subjects would be ideal.

Portuguese Man o' War (oceana.org)

The Bugs Of The World Could Squish Us All

Swarm intelligence, compound intelligence, or group intelligence may emerge as an important concept as AI develops toward higher complexity.  Whether these terms should be considered synonymous is doubtful.

Compound features in biology are the result of control in differentiation during the development of an organism from a single cell.  Compounding in biology is a single function performed across like elements.

Swarms are a result of distinct organisms operating in proximity.  Swarming in biology is similarity in complex independent behaviors that appear coordinated but are generally engaged in as a defense from predators.

Group intelligence may be distinct from compound intelligence in that adjacent units may be in agreement or opposition, as can be case with intelligent beings in groups.  The agreement in the group that a variety of models is permissible.  We call them opinions, but they are distinct matches of models to problems that produce different projections and suggest different selections from among the group's options.

The terms general and strong appear in this question and many others and it appears that, mainly for historical reasons, may continue to frustrate clarity.

All intelligence is general in that it learns a generality from specific experiences and then applies that generality to future scenarios to achieve objectives within those scenarios.  What makes the application of the generality intelligent is that it is expected to work because it has been working for similar scenarios.

All intelligence is specific in that it is limited to scope of what generalities have been discovered.  Certain techniques are significantly more general that others because they are allegedly domain independent.  We call this mathematics.

Consequently, the development of artificial intelligence is not a path from specific to general but one of discovering generalities and applying them to specifics.  One could say that the most general thinking of mathematics applied to the computer is the primary activity of applied artificial intelligence, and the capabilities that emerge are a more sophisticated as more sophisticated mathematics is represented in working software and hardware.

The generalities take on greater complexity so that they can apply to a greater number of specific scenarios.  That is not a gain in strength but a gain in the breadth of potential application.

Physalia physalis is a symbiotic colony of organisms of four types, the pneumatophore (or float), dactylozooids (long tentacles), gastrozooids (feeding tentacles), and gonozooids which produce reproductive gametes.  To avoid breaking the historic conception of animals, these organisms are called polyps.  The venom of the colony does not come from any of those organisms but rather from another symbiotic organism, cnidocytes, which attach to the tentacles and are released under strictly controlled conditions.

In addition to all this symbiosis, there are several species of fish that use the colony for shade and protection that the colony lets swim among the tentacles and that have formed partial immunity to the cnidocytes.  These complex symbiotic networks are not fully understood, but they seem to operate well and create sustainable inter-specie systems.

It's not likely that the organisms, within their lifetimes adapt or remember, but the DNA of each organism has involved in a way that resembles intelligence in that the colony and its symbionts have adapted to the ocean's surface and its biology.

The idea that the system of the biosphere is the first example of broad intelligence is likely, in that design excellence has emerged from an evolutionary processes.  It is not altogether ridiculous to propose that human intelligence is a higher speed approach in neurons to the slower speed of DNA replication in larger organisms.  Because of the metabolic requirements of growth slows evolution for larger organisms with a greater cell count, these larger organisms may have needed to develop a way to achieve the nimble adaptivity of their lower ancestors.

Neurology facilitates the approximation of some aspects of evolution and may have been the most attainable natural solution to reacquire nimble adaptivity.

Attempting to apply these various ideas to the current and ongoing development of autonomous vehicles reveals a gap in understanding.  We don't yet have the mathematics developed to understand how compound, swarm, or group intelligence can be used in the laboratory to accomplish well defined problems in controlled execution scenarios.  That is probably a prerequisite to using these ideas in vehicle control.

The system designs of future cars may be like a colony of independent or semi-independent components that each have a role and purpose.  Compound, group, and symbiotic designs are likely to develop.  The current activities of vehicles on the road or in the air near airports is like a swarm, so the application of that idea is obvious: Avoid collisions.

I did work on compound intelligence because that is the direction that Google is trying to go. I couldn't find any basis for it. In other words, having a collection of AI expert systems does not seem to provide any collective intelligence. You would also need some kind of control program that could decide which system to use. Currently, Google relies on the user to choose. If Google was able to create an independent control program it would already be out. This does not seem to be a matter of complexity or code tweeking but a fundamental limit with AI.


Counterexamples to the reward hypothesis On Sutton and Barto's RL book, the reward hypothesis is stated as

  that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)

Are there examples of tasks where the goals and purposes cannot be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal?

All I can think of are tasks with subjective rewards, like "writing good music", but I am not convinced because maybe this is actually definable (perhaps by some super-intelligent alien) and we just aren't smart enough yet. Thus, I'm especially interested in counterexamples that logically or provably fail the hypothesis.

What if a scalar reward is insufficient, or its unclear on how to collapse a multi-dimensional reward to a single dimension. Example, for someone eating a burger, both taste and cost are important. Agents may prioritize taste and cost differently, so its not clear on how to aggregate the two. It is also not clear on how a subjective categorical taste value can be combined with a numerical cost.

The book sets this hypothesis up by laying out a few assumptions:

In reinforcement learning, the purpose or goal of the agent is formulated in terms of a special signal called the reward, passing from the environment to the agent. At each time step, the reward is a simple number.

We could think about what counterexamples to those assumptions might be:

The reward signal originates internally, instead of originating from the environment. (e.g. meditation, or abstract introspection)
The signal is not received every time step, or isn't necessarily expected to be received at all. (e.g. seeking of transcendent experiences)

What might be common for these counterexamples is that the reinforcement learning mechanism itself undergoes spontaneous change. A signal that would have been positive before the spontaneous change might now be negative. The reward landscape itself might be completely different. From the agent's perspective, it might be impossible to evaluate what changed. The agent might have a 'subconscious' secondary algorithm that introduces changes in the learning algorithm itself, in a way that's decoupled from any reward-defined behavior.


Nonbinary and binary values in input tensor In my input tensor, I would like to use both integer values as well as booleans. For example, if there is a spelling difference between 2 texts, I want to set the value to true, and otherwise false. In the same tensor, I would like to assign a value to, for example, the maximum number of consecutive messages, which will be an integer.

Am I allowed to use 0's and 1's for the booleans together with integers, or will it have any negative impact on the working of the network? The ANN wont see any difference between the binary and nonbinary values, but is it a problem?

If the two inputs for the binary values are held to the domain set $\{0, 1\}$ during training, testing, and use, it will not break the network functionality, although the question is a good one. Why use so many bits to hold one?

Theory of Holors: A Generalization of Tensors, by Parry Moon and Domina Spencer (ISBNs 978-0521019002 and 0521019001), propose holors precisely because of this kind of limitation on the natural heterogeneity of numerical structure. Gibbs, Ricci, Einstein, and others used holors in their mathematical expressions but without the name holor. Holors are are conceptually related to objects in ontology and classes in object oriented design, but they are numbers, so they fit into mathematical expressions as scalars, vectors, matrices, vector fields, and other tensors do.

This is the basis for Coplien's operator overload in C++ and one of the reasons the language is too mutable to be mastered by average applications programmers. Support for what Moon and Spenser called holors is also the basis of decoupled type safety in some of the early LISP object oriented frameworks.

The current programming paradigm we seen in Java, Python, and other popular languages was set in FORTRAN, which was more attainable for average programmers, so we are stuck with homogeneous structure, even in AI libraries and GPUs. Therefore, once one number requires an IEEE 64 float, bytes and bits will use all 64.

In artificial networks, this only matters in the very first multiplication with a parameter.


Is it possible to have a variable-length latent vector in an autoencoder? I'm trying to have a simple autoencoder but with variable latent length (the network can produce variable latent lengths with respect to the complexity of the input), but I've not seen any related work to get idea from. Have you seen any related work? Do you have any idea to do so?
Actually, I want to use this autoencoder for transmitting the data over a noisy channel, so having a variable-length may help.

If you use RNNs, then I think the solution is to use padding (zero padding) with max sequence length (that is the max number of words in a text) in order to tell your model to skip the zeros when possible. In that way, your model will try to learn a good representation of your input with fixed size. If you do not know this dimension, a solution may be to grid search this hyperparameter.
If you still want to exploit the dimensionality difference, maybe you can train different models with fixed size dimension of the representation dependently of the dimension of the input. That is, for example, use one for small, one for medium and one for large dimensions, but this should surely require to have a large and quite balanced initial dataset.
Another idea could be to use the autoencoder with a fixed latent dimension. Then, do effective clustering on your samples using their latent representation, considering that similar representations should have similar dimensionality requirements (?). After that, you could train your initial dataset on k models, the same number as the clusters. That is, there should be k different latent spaces. The goal is to match each instance to the correct model. At first, you should train them all with each instance, but as the training progresses, you should maybe go with binary search for each instance in order it to find the correct model, assuming that there is total order in the measuring of the dimensionality requirements. Of course, this is just an idea, I don't know if it is going to be really helpful at all.

You might want to look at an encoder-decoder sequence to sequence model. This model allows you to input and output data with variable length.


Can an AI be trained to generate the outline of a story? I know that one of the recent fads right now is to train a neural network to generate screenplays and new episodes of e.g. the Friends or The Simpsons, and that's fine: it's interesting and might be the necessary first steps toward making programs that can actually generate sensible/understandable stories.

In this context, can neural networks be trained specifically to study the structures of stories, or screenplays, and perhaps generate plot points, or steps in the Hero's Journey, etc., effectively writing an outline for a story?

To me, this differs from the many myriad plot-point generators online, although I have to admit the similarities.  I'm just curious if the tech or the implementation is even there yet and, if it is, how one might go about doing it.

As far as I am aware, this has not been done yet.

I see several problems with this. A neural network is basically a classifier, which matches an input to an output. Both input and output are usually numerical values, though they could be matched to concepts or words.

To train a NN you provide an appropriately encoded input, and the corresponding output. The NN learns the associations between the two, and can then classify unseen input accordingly. This has recently been used to transform images in a particular style etc.

What would the input and output be for generating screenplays? You could use previous scripts as inputs, but what would the output be? It could be narrative 'moves' of some sort, perhaps. So you could train an NN to recognise narrative elements from screenplays.

However, you are still not creating anything, but just recognising stuff. You would need some other input. I guess you could train an NN on "The Simpsons", get a narrative structure, and then present it with an Episode of "Friends" and see what happens. It won't be a new episode of a screenplay, though.

The other way round might work: you feed it narrative moves (a kind of story skeleton), and get a script out. But it would need a lot of (human) post-editing to be at all useful.

I think an NN is the wrong tool to use here. There has been work done with generating stories and screenplays, even way back in the early days of AI. But that was all based on symbolic AI, not on the kind of ML which seems to currently be en vogue. Have a look at James Ryan's website; he has recently written an overview over historic approaches to story (and screenplay) generation.

As far as I know, there isn't any system like you describe yet. However, there are some interesting approaches to narrative intelligence that can be found at the University of New Orleans Narrative Intelligence Lab site: https://nil.cs.uno.edu/

Hopefully those can be helpful in guiding a deep-learning approach to narrative generation problems.


Can a technological singularity only occur with superintelligence? In Chapter 26 of the book Artificial Intelligence: A Modern Approach (3rd edition), the textbook discusses "technological singularity". It quotes I.J. Good, who wrote in 1965:

  Let an ultra-intelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultra-intelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind. Thus the first ultra-intelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

Later on in the textbook, you have this question:

  26.7 - I. J. Good claims that intelligence is the most important quality, and that building ultr- intelligent machines will change everything. A sentient cheetah counters that "Actually speed is more important; if we could build ultrafast machines, that would change everything" and a sentient elephant claims "You're both wrong; what we need is ultrastrong machines," What do you think of these arguments?

It seems that the textbook question is an implicit argument against I.J. Good. Good may be treating intelligence as valuable, simply because man's strengths lies in that trait called "intelligence". But other traits could be equally valued instead (speed or strength) and sentient beings may speculate wildly about their preferred traits being "maximized" by some machine or another.

This makes me wonder whether a singularity could occur if we had built machines that were not maximizing intelligence, but instead maximizing some other trait (a machine that is always increasing its strength, or a machine that is always increasing its speed). These types of machines can be just as transformative - ultrafast machines may solve problems quickly due to "brute force", and ultrastrong machines can use its raw power for a variety of physical tasks. Perhaps a ultra-X machine can't build another ultra-X machine (as I.J. Good treated the design of machines as an intellectual activity), but a continually self-improving machine would still leave its creators far behind and force its creators to be dependent on it.

Are technological singularities limited to ultra-intelligences? Or technological singularities be caused by machines that are not "strong AI" but are still "ultra"-optimizers?

That would be a no for speed or strength, if you have a super strong entity but it cannot research new materials, it will be quickly limited, same thing for speed, Basically, you need something out of their field to improve them, which makes a runaway improvement impossible.

Though, we already have super strong and super fast machines, those are cars, trucks, hydraulic presses, industrial exoskeletons etc... But, even though we can build better ones through the use of the old ones, we still need to research stuff that can't be improved by old ones.

What we need for a singularity is a field where an improvement in it makes further improvement easier. And I don't know a field where this doesn't involve intelligence.
If there is one, that may be possible to have a non intelligence driven singularity there.

This May not be what you were looking for,  but technically yes. Although not for Speed and Strength. But you could randomly guess new Mathematical/Physical/chemical solutions to become more efficient in random guessing (basically anything that allows the machine to compute faster and to maybe simulate the effect of those findings) thus technically achieving something similar to a singularity, without having to have any Intelligence at all actually (or just on a human Level), since you could just brute force all.

Is this efficient? No, not even close to being in any way feasible.
Does it work? Technically, Yes.

It would be a singularity of sorts, since it improves itself continuously, but it wouldn‘t need to improve its own intelligence.

Of course, some findings might make it possible to become more intelligent, but let‘s just assume it doesn‘t apply those findings to itself.


Are there any easy ways to create annotated training images for object detection? For the purposes of object detection, are there any easy ways to create annotated training images? For example, if we have $10,000$ images and want to draw bounding boxes on 2 objects for each image, do we have to physically draw those boxes? Is that what most people do these days to create training data?

From what I read in the papers and my experience, the trend lately is to generate synthetic data using a photorealistic 3D graphics engine for automating the image labelling. The process would be:

Get a visually realistic 3D engine such as Unreal, Unity... any videogame engine would work
Build 3D models of the objects you want to detect (or ask for help to a 3D artist)
Build a 3D environment for placing the objects 3D models
Generate random instances of the objects in the environments with different environmental conditions: lighting, brightness, position, rotation...

Since you have the 3D model of the object and the whole control in the 3D engine you can annotate images, bounding boxes, pixels... or whatever stuff you want in an automated way.

This is an approach for automatic image labeling with synthetic data. Automating the labeling of real data is a challenge yet to be tackled.


How much can the addition of new features improve the performance? How much can the addition of new features improve the performance of the model during the optimization process?

Let's say I have a total of 10 features. Suppose I start the optimisation process using only 3 features. Can the addition of the 7 remaining ones improve the performance of the model (which, you can assume, might already be quite high).

It depends on the used network as well as the feeding mechanism but let's give an example;

When working with LSTM, giving the time data (as an integer sequence) in addition to the time-series data(coming from features) dramatically increases the performance of the network.

[$X_{0}$,$X_{1}$, ...] $\rightarrow$ [[$X_{0}$,$t_{0}$],$[X_{1}$,$t_{1}$], ...]

If you go and look for the kaggle competition winner's notebooks, they do also create additional features based on the featured data.

Let's assume that the performance is already quite high on the three features so that you can predict those three features with high reliability.
It would only make sense to increase the number of features if you would like to predict additional features!

The optimization of 10 features, as opposed to optimization of 3 features, will converge more slowly.

  Let's say I have a total of 10 features. Suppose I start the optimisation process using only 3 features. Can the addition of the 7 remaining ones improve the performance of the model (which, you can assume, might already be quite high).

The answer might be no.

The accuracy and reliability of the convergence toward ground truth (a formalized objective used to guide the optimization process) when split into groups of three and seven features may be better or worse than when left as a group of ten.
Except in rare cases, the results will not be the same.  The likelihood of identical results is so low it may never happen in the world in the next century except when conditions are arranged solely to cause it to occur.

Why then do may approaches group the dimensions of the result and converge on groups of axes, then another, then another, and back again to the first, iterating until the convergence goal is reached?  This approach is used to reduce the time and computing resources used to reach the optimal.  As the problem complexity increases, the use of groupings in this way is more common.


How can auto-encoders compute the reconstruction error for the new data? Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous if the new data has a large reconstruction error, i.e. it was hard to fit the features as in the normal data.
Even if the training is supervised by learning to reconstruct the same data, how is the reconstruction error computed for the new data?

It is computed just like in training. You take an MSE or something along these lines between the input and the output. You set a threshold for it. If new data's reconstruction error is higher than your threshold, then it is anomalous otherwise it isn't.

By Unsupervised they refer to the learning process, which is Unsupervised. Although, it does not appear so, but Unsupervised loosely means you work with data points and data points only without the use of any information about those data points.

I do not have idea about how AE's are used for anomaly detection, but from the point of view of AE's, I will explain what might be happening.

In AE's the hidden representations (if hidden units are less than visible units) the weights for the encoder network are not independent of each other, thus the values coming out of a hidden node after encoding operation are not independent of each other, which if speaking mathematically will mean each hidden node is modelling complex probabilistic interactions between the input features. Thus, the AE encoder learns that if the given feature $x_1$ is of this magnitude, it also expects $x_2,...x_n$ to be of certain other magnitudes. Similarly, the decoder also learns how to decode this complex interactions to original data.

The point to note is that in AE's with hidden nodes less than input nodes, if encoder function is given by $e$ and decoder by $d$ then the operation you are doing is $$d(e(x))=x$$ and not $$e^{-1}(e(x))=x$$

Decoder is reversing the inputs as "learned" and "generalized" from inputs, and not by "learning" to perform the inverse operation (this might be the case where $nodes_{hidden}$ $\geq$ $nodes_{input}$).

Now, say there is an anomaly. The encoder, outputs something which the decoder will see does not match the way it should match with other hidden nodes output (normally if hidden node1 is outputing $h_1$ the decoder expects $h_2$ from hidden node 2, but now the decoder is getting $h_1$ but not getting $h_2$ instead getting $h_2 + \epsilon$ where $\epsilon$ is noise and as a result fails to understand and decode the hidden representation properly.


Why do neural networks trained on identical datasets and with identical hyper-parameters have different performances? I found that fully connected neural networks trained on identical data sets with identical hyper-parameters can have different performances or accuracies (7-8% of deviation). Is this an unusual phenomenon? What is the cause? What is an acceptable deviation?

This should simply be due to the fact that network weight initialization is random.

but at different times

you have a variance problem, I suggest to study basic machine learning and the bias-variance tradeoff

The intuition: the model has a large capacity with a little or no regularization, therefore the model has a large degree of freedom of fitting the data.

  What is an acceptable standard deviation level

+/- 0.05

  Is that an unusual phenomenon

some insights from: Optimal Hyperparameters for Deep LSTM-Networks for Sequence
Labeling Tasks


What is the advantage of using cross entropy loss & softmax? I am trying to do the standard MNIST dataset image recognition test with a standard feed forward NN, but my network failed pretty badly. Now I have debugged it quite a lot and found & fixed some errors, but I had a few more ideas. For one, I am using the sigmoid activation function and MSE as an error function, but the internet suggests that I should rather use softmax for the output layer, and cross entropy loss as an error function. Now I get that softmax is a nice activation function for this task, because you can treat the output as a propability vector. But, while being a nice thing to have, that's more of a convinience thing, isn't it? Easier to visualize?
But when I looked at what the derivative of softmax & CEL combined is (my plan was to compute that in one step and then treat the activation function of the last layer as linear, as not to apply the softmax derivative again), I found:
$\frac{δE}{δi}$ = $t$ − $o$
(With $i$ being the input of the last layer, $t$ the one hot target vector and $o$ the prediction vector).
That is the same as the MSE derivative. So what benefits does softmax + CEL actually have when propagating, if the gradients produced by them are exactly the same?

Short answer: larger gradients
That is not the derivative of the softmax function. $t - o$ is the combined derivative of the softmax function and cross entropy loss. Cross entropy loss is used to simplify the derivative of the softmax function. In the end, you do end up with a different gradients. It would be like if you ignored the sigmoid derivative when using MSE loss and the outputs are different. Using softmax and cross entropy loss has different uses and benefits compared to using sigmoid and MSE. It will help prevent gradient vanishing because the derivative of the sigmoid function only has a large value in a very small space of it. It is similar to using a different cross entropy loss where the combined derivative of the loss and sigmoid is $t - o$.
Information on derivatives of cross entropy with sigmoid function and with softmax function. I would also suggest some more research on cross entropy loss functions beyond my links.

If you look at the definition of the cross-entropy (e.g. here), you will see that it is defined for probability distributions (in fact, it comes from information theory). You can also show that the maximization of the (binomial/Bernoulli) log-likelihood is equivalent to the minimization of the cross-entropy, i.e. when you minimize the cross-entropy you actually maximize the log-likelihood of the parameters given your labelled data. Hence the use of the softmax is theoretically founded.
Regarding the supposed derivative of the cross-entropy loss function preceded by the softmax, even if that derivative is correct (I didn't think about it and I don't want to think about it now), note that then $t - o$ is different depending on whether $o$ is a probability vector or an unnormalized vector (which can take arbitrarily large numbers). If $o$ is a probability vector and $t$ a one-hot encoded vector (i.e. also a probability vector), then all numbers of $t - o$ will be in the range $[-1, 1]$. However, if $o_i$ can be arbitrarily large, e.g. $o_i = 10$, then $t_i - o_i \in [-10, -9]$. So, the propagated error would be different if $o$ was not a probability vector.


What are the main problems hindering current AI development? I have a background in Computer Engineering and have been working on developing better algorithms to mimic human thought. (One of my favorites is Analogical Modeling as applied to language processing and decision making.) However, the more I research, the more I realize just how complicated AI is.

I have tried to tackle many problems in this field, but sometimes I find that I am reinventing the wheel or am trying to solve a problem that has already been proven to be unsolvable (ie. the halting problem). So, to help in furthering AI, I want to better understand the current obstacles that are hindering our progress in this field.

For example, time and space complexity of some machine learning algorithms is super-polynomial which means that even with fast computers, it can take a while for the program to complete. Even still, some algorithms may be fast on a desktop or other computer while dealing with a small data set, but when increasing the size of the data, the algorithm becomes intractable.

What are other issues currently facing AI development?

we don't really know what intelligence is.
we don't truly understand the best model of intelligence we have available (human intelligence) works.
we're trying to replicate human intelligence (to some extent) on hardware which is quite different from the hardware it runs on in reality.
the human brain (our best model of intelligence) is mostly a black-box to us, and it's difficult to probe/introspect its operation without killing the test subject.  This is, of course, unethical and illegal. So progress in understanding the brain is very slow.

Combine those factors and you can understand why it's difficult to make progress in AI. In many ways, you can argue that we're shooting in the dark. Of course, we have made some progress, so we know we're getting some things right. But without a real comprehensive theory about how AI should/will work, we are reduced to a lot of trial and error and iteration to move forward.

One obstacle to the development of AI is the fundamental limitations of computer memory. Computers, at a fundamental level, can only work with bits. This limits the type of information that they can describe.

EDIT:

The precise nature and complexity of human memory isn't fully understood, but I would argue that at the very least, human memory is well adapted for the types of tasks that humans perform. Thus, computer memory, even if theoretically capable of representing everything that human memory can, is  probably inefficient and poorly structured for such a task.


Why is no activation function used at the final layer of super-resolution models? I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer.
What's the rationale behind that?

I am not into the field of super-resolution, but I think this question applies to general neural network construction.
Usually, you try to solve a classification problem or a regression problem with your neural network.

For classification, you try to predict probabilities that a specific output corresponds to a specific class. Therefore, every output value should be a probability and therefore have a range between 0 and 1. To achieve this, you usually use a softmax or sigmoid function as your last layer to squash the output between 0 and 1. In addition to this (which is wanted in classification tasks), these functions raise the probability output of likely classes while decreasing the probability of all other unlikely classes (therefore enforcing the network to choose for one specific class over the others).

For the regression task, you are not looking for probability values as your output values but instead for real-valued numbers. In such a case, no activation function is wanted, since you want to be able to approximate any possible real value and not probabilities.

So, in the case of super-resolution, I think the generated output is a map where each value corresponds to a pixel value of the super-resolution image. In that case, your pixels are real number values and no probabilities. So, you are solving a regression problem.
But you could also go with a classification approach, where you have 256 output maps that give probability to each possible pixel value between $0$ and $255$.

As discussed here:

https://www.researchgate.net/post/What_should_be_my_activation_function_for_last_layer_of_neural_network

Linear is the preferred activation function.

But then linear activation function is equivalent to no activation function at all:

https://datascience.stackexchange.com/questions/13696/lack-of-activation-function-in-output-layer-at-regression


Why do the inputs and outputs of a convolutional layer usually have the same depth? Here's the famous VGG-16 model.

Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth?  What's the reason for that?
Is there a theory or paper trying to explain this kind of setting?

Keeping the same channel size allows the model to maintain rank but i would say the main reason is convenience. Its easier book keeping.

Also in many model cases output features need some form of alignment with the input (example being all models using residual units -- $\hat{x} = F(x) + x$


Is reinforcement learning needed to create Strong AI? By reinforcement learning, I don't mean the class of machine learning algorithms such as DeepQ, etc. I have in mind the general concept of learning based on rewards and punishment.

Is it possible to create a Strong AI that does not rely on learning by reinforcement, or is reinforcement learning a requirement for artificial intelligence? The existence of rewards and punishment imply the existence of favorable and unfavorable world-states. Must intelligence in general and artificial intelligence in particular have a way of classifying world-states as favorable or unfavorable?

Motivational Theory Versus Learning Theory

Rewards and punishment are based on human motivational theory and are broad generalizations of the neurological factors in mental growth.  At a neurological level, there are chemically based signals, such as seratonin, dopamine, and oxytocin.  Some of these have their own pathways in the brain.

There are also pain signals arising from the body and internally generated stimulatory and inhibitory signals in chemical and electro-chemical forms.  There is neuroplasticity, where the brain circuits form new topologies.  The hippocampus is a structure known to be important in learning.1  The study of learning in the brain is fervent, but the object of study is complex.  Even so, every year, progress is made.

Multiple Dimensions or Singly Quantifiable Intelligence

Conjecture about strength versus weakness of intelligence may be of limited use.  There are two valid criticisms discussed in other Q&A here in more detail.

Twenty-two independent genes have been identified that correlate to typical academic intelligence quantification in humans, rational and linguistic, eleven of which were identified in 2017.2  There are likely more.  Creative, athletic, emotional, and leadership qualities have not yet been captured in standardized intelligence testing, so it is likely that additional genes related to those abilities exist.  The notion that intelligence can be accurately aggregated into a single g-factor is weakly supported, if it were true, implies that all twenty-two genes affect only one magical function.  Statistically, such alignment is close to impossible.
Intelligence is only meaningful in context.  Fast learning in one context appears like mental brilliance, but in another context the same learning capability may be dysfunctional.

Reinforcement in the Larger Pool of AI Research

Reinforcement Learning is not a requirement for AI as far as is currently known.  Reinforcement is a mathematical strategy for driving a set of behavioral parameters toward a defined optimum.  Although its variants work well for many applications, it is not closely aligned with any of the features of the hippocampus, the limbic system, the cerebral cortex, or any other learning systems in the human brain.

Even though there is still much to learn about brain workings, the success of reinforcement learning, rule based systems, convolution, attention based networks, and generative networks provides information about what is possible in computers and what may or may not be a mechanism of the brain.

What Reward and Punishment Imply

The existence of rewards and punishment imply the existence of favorable and unfavorable family, workplace, or community states.  If the term world in the question is referring to the immediately related environment, then the sentence is probably true, but it may be misleading without clarification.

World states are based on economics which involve the minds of many people in community, aggressive actions, resource acquisition, consumption, cultural perspectives, and many other factors.  A single individual's perception of world states is subjective.

Defying Motivational Mechanisms and Intelligence

The same person on two different days may have different ideas of what is favorable or unfavorable, which may or may not correlate to reward and punishment motivations.  The words rebellion and indifference describe states where external reward and punishment produce opposing or independent results in learned behavior respectively.  Some of the most intelligent behavior in humans were of these kinds.

What drives the classification of external states within a single individual is something not well understood.  The neurological factors involved in the emergence of opposition to pain, pleasure, reward, and punishment is an interesting area of study and certainly related to AI.

Genetic factors involved in such contrary learning are likely to fall outside the twenty-two genes, since those genes correlated with standardized academic testing.  Academia is about conformance.  If we don't agree with the author of the textbook for a course, we pretend we do when we take the test if we want a high grade.

World-changing intelligence, such as that of Ghandi's nonviolent opposition, Isaac Newton's invention of physics, the furtherance of logical disproof by Socrates, or whoever dared to learn how to light a fire. is always the exhibition of classifying in a way that contradicts that of the surrounding culture.

Discovery and Acceptance in Group Learning

This is even true of project-changing intelligence.  A software engineer might say, "That software will never work because the database schema indicates a one to many relationship between companies and office addresses, but it is a many to many relationship."  Such a statement doesn't change the world and may completely lack group acceptance when spoken.  Later in the project, after the statement is fully accepted, the reclassification changes the design so that the project is not caught in a well of wasted coding.

Interestingly, convincing the others often falls to someone other than the originator of the idea — someone else on the team that has a different set of mental abilities.

It is true that classification of what is favorable or unfavorable is key, but how that classification exhibits intelligence is not well understood.

What Benefits AI Development

Although not a requirement, a solid approach to AI development is discovery, from the directions of multiple fields of study, of what intelligence is.  Some say we know what it is now, but most definitions are largely expressions of historical inaccuracies.

Clarity about what intelligence is and what it is expected to be in such a way that it can be represented mathematically is needed.  Only then can we better achieve it in humans and machines.  Sources of understanding may come from discovery in any area of study, such as these.

Evidence based psychology
Evidence based genetics
Evidence based neurological research
Evidence based artificial systems design

There is a growing trend to hack AI, but playing with AI as if it were a game might be a formula for creating Frankenstein.  A useful and collaborative collection of AI devices in robots, autonomous vehicles, and data centers will require interdisciplinary inquiry, experimentation, and engineering.  The description of this site implies that ethics should be involved, and that's a smart conception.

With a purview that crosses divisions in study and practice, central themes and models of intelligence are likely to develop.  This question is on the right track in this respect. It asks questions without accepting too quickly conjecture  — ideas and hypotheses presented but not yet shown to be true.

What is Known

That computer simulation or replication of human abilities have already been achieved and that such achievements indicate AI to be a reasonable long term objective few would deny.  Certainly in the areas of arithmetic, high speed machine control, sorting of addressed parcels, and many other roles, computers far outperform humans.  Whether world-changing intelligence will emerge in artificial systems is hoped for by some, feared by others, and asserted as inevitable by those who seek funding to prove it.

References

[1] Training your brain: Do mental and physical (MAP) training enhance cognition through the process of neurogenesis in the hippocampus?, D.M. Curlik and T.J. Shors, 2012

[2] Genome-wide association meta-analysis of 78,308 individuals identifies new loci and genes influencing human intelligence, Suzanne Sniekers et. al., 2017

It's impossible to give a definitive 'yes' answer to your question, since that would require proving that alternatives cannot exist.

More philosophically, it depends on what you mean by "preference over world states":

However counter-intuitive it might seem, it is conceivably possible to create Strong AI purely from local condition-action rules, in which there is no global concept of 'preference value' and/or no integrated notion of 'world state'.


What are the criteria for a system to be considered intelligent? For example, could you provide reasons why a sundial is not "intelligent"?
A sundial senses its environment and acts rationally. It outputs the time. It also stores  percepts. (The numbers the engineer wrote on it.)

What properties of a self driving car would make it "intelligent"?

Where is the line between non intelligent matter and an intelligent system?

Typically, I think of intelligence in terms of the control of perception. [1] A related, but different, definition of intelligence is the (at least partial) restriction of possible future states. For example, an intelligent Chess player is one whose future rarely includes 'lost at chess to a weaker opponent' states; they're able to make changes that move those states to 'won at chess' states.

These are both broad and continuous definitions of intelligence, where we can talk about differences of degree. A sundial doesn't exert any control over its environment; it passively casts a shadow, and so doesn't have intelligence worth speaking of. A thermostat attached to a heating or cooling system, on the other hand, does exert control over its environment, trying to keep the temperature of its sensor within some preferred range. So a thermostat does have intelligence, but not very much.

Self-driving cars obviously fit those definitions of intelligence.

[1] Control is meant in the context of control theory, a branch of engineering that deals with dynamical systems that perceive some fact about the external world and also have a way by which they change that fact. When perception is explicitly contrasted to observations, it typically refers to an abstract feature of observations (you observe the intensity of light from individual pixels, you perceive the apple that they represent) but here I mean it as a superset that includes observation. The thermostat is a dynamical system that perceives temperature and acts to exert pressure on the temperature it perceives.

(There's a philosophical point here that the thermostat cares directly about its sensor reading, not whatever the temperature "actually" is. I think that's not something that should be included in intelligence, and should deserve a name of its own, because understanding the difference between perception and reality and seeking to make sure one's perceptions are accurate to reality is another thing that seems partially independent of intelligence.)

Intelligence is the efficiency of an action in serving some purpose.

Both sundials and self-driving cars are intelligent systems.

Anything that serves some purpose exhibits intelligence.

One thing is more intelligent than another thing if it achieves some purpose in less steps.


Can we implement GAN (Generative adversarial neural networks) for classication problem like Fraud detecion? Problem: Fraud detection
Task : classifying transactions as either fraud/ Non-fraud using GAN

Generative networks was the first popular class of topologies that had compound feedback, corrective signaling at more than one layer.  In popular network designs such as MLPs (multilayer perceptrons), CNNs (convolutional neural networks), and LSTM (long short term memory) networks, the backpropagation is a single layer mechanism.  The mechanism distributes corrective a signal creating a closed loop that ideally converges to an optimal network behavior.

The speed, accuracy, and reliability of convergence of networks in an AI system depend on a number of engineering choices at a lower level of design and development.

Selection of algorithm variants and extensions
Selection of data set
Drawing of the training samples
Data normalization
Initialization
Assignment of learning rate or learning rate policy
Other hyper-parameters

Generative networks are designed to achieve a higher level balance between two networks, each of which have their own backpropagation, and thus have a compound feedback topology.

Considering the original GAN approach in the context of fraud detection, a discriminative network $D$, in a way, detects fraudulence originating from the generative network $G$, and, in a way, $G$ learns authenticity from the corrective signal provided by $D$, even if the authenticity learned is superficial.  However, in the case of credit card fraud detection from a numerical set of data, the fraudulence is not internal to the system to be designed as in the case of GAN.  Without both the $G$ and $D$ networks, the design is not a generative network.

Since the comment stated, "I have to explore GAN for this problem," the key question is this.

  If this system is to benefit from a GAN, what would the GAN generate that could contribute to the solution?

If the answer is nothing, then a GAN is not the correct approach.  A discriminative network without a generative network is not a GAN.  If this question is from an assignment to train a network using a static and already existing data set, a MLP may suffice, depending on the expectations of the teaching staff.  If that is the case, the goal will be to analyze the data set using statistical tools to determine if it is already normalized well enough to lead to excellent convergence accuracy, speed, and reliability.  If normalization is indicated, find ways to present the data to the input layer of the MLP in such a way that each input is relevant and has a well distributed range of values across the data type.

Ensure there is no data loss in this process, and remove redundancy where it can easily be removed.  By redundancy removal is meant, if one of the dimensions of data have only four possible values of between 10 and 1,000 bytes, encode the four possible values in two bits because to do so removes unnecessary burden from the training.

Similarly, the output layer must be chosen to produce an encoded set of possible output values and the decoding must be written to produce usable output.  In this case, it appears that the labels are, Fraudulent and Authentic.  Which means that the last layer would be a single binary threshold cell.

For this simple MLP approach and for many of the approaches mentioned next, there are examples available in Python, Java, C++ and other languages from which implementation can be initiated.  If the design must stand up to a more realistic fraud detection scenario, either for credit cards, junk mail, banking, log-in, or communications those experienced with fraud detection know that the above approach is entirely insufficient except as a starting point for learning.  In such a case, these considerations further complicate the design of the solution.

Fraudulence and authenticity is not necessarily sufficient information for investigators.
    The likelihood of fraudulence and the reason why fraudulence is suspected is also pertinent for end users of production output,
    so that security holes can be blocked and investigation may be directed toward prosecution or some other set of remedies.
Fraudulence is usually a security breach phenomena that involves continuous improvement in the attacks until the attackers
    are located and their work is interrupted by arrest.
    What initially works as fraud detection may not work a week, a day, an hour, or a minute later.
Fraud detection, if it to be useful in a real financial context, must be part of a fraud prevention system,
    and training on a static data set may be meaningless in that larger usage environment.

In AI systems that work in real production environments, it may be worth investigating using an LSTM network in combination with a reinforcement learner.  Such may be over a beginner's level of design and implementation experience, but using either an LSTM network or a basic Q-learning approach instead of both of them together may be a reasonable starting point for dedicated students.  If the answer to the question about what a GAN could generate that would be of use must be yes, the following ideas may be helpful.

A generative network could generate transaction data with a distribution that resembles authentic transactions, and such has been successful in the past.
A generative network could generate transaction data with a distribution that resembles attacks on an account by a fraudulent user. That has been done too.
A generative network could theoretically generate hacker agents, although this may not have yet been done.
A generative network could be used to generate patterns of authentic transactions based on some authenticity measure other than the data set mentioned, and the trained discriminative network could then be employed in some way in a fraud detection system in production, although it is unclear what advantage would be gained.

The reason a GAN was suggested would need to be clarified before this answer could provide a more exact direction for AI system topology and project approach.

Yes, although the traditional technique is to use Anomaly Detection for this sort of problem. The reason for that is that the space of possible frauds is very much larger than the space of legitimate transactions and so it is difficult to model the former. The usual idea is to thoroughly explore the smaller space of legitimate transactions and flag as fraudulent anything outside of that space.
Using the GAN method could be interesting as it might identify novel fraud techniques, and those are always useful to know.


Traveling salesman problem variant: which algorithm to choose? I have an industrial problem which I'm trying to cast as a Traveling Salesman problem (TSP) in 3D euclidian space. There are physical limitations which implies that some subpaths may or may not be valid based on simple rules.

What algorithm is best to deal with the TSP given that there are rules/model/constraints?

It could be done with Genetic algorithms for example, but the only way i see how to incorporate those rules is by including them somehow within the fitness function. But i feel there should be more suitable approaches.

Would reinforcement Q-learning or other algorithms be more appropriate for a rule-based euclidian TSP?

If you have lots of rules then you should be able to come up with lots of heuristics for A*. If I were you I would try A* first, and come up with as many heuristics as I could. You can also use Deep Q learning. I don't think you want to just throw coordinates into a Deep Q network. You want a simpler representation. I would probably give the nodes symbolic identifiers like sparse matrices, integers or sigmoids, and then craft the loss function to reflect the actual cost of moving from node to node and ascribe a high cost to illegal moves.


How to use a Generative Adversarial Network to generate images for developmental analysis? I want to generate images of childrens' drawings consistent with the developmental state of children of a given age.  The training data set will include drawings made by real children in a school setting.  The generated images will be used for developmental analysis.

I have heard that Generative Adversarial Networks are a good tool for this kind of problem. If this is true, how would I go about applying a GAN to this challenge?

A generative adversarial network is probably not the best approach for generating the images desired.  We can assume from the comments that the data is not collected.  That's a good thing, because a set of rasterized images, labeled with student age or grade is an inferior input form.

It appears that access to a student population is planned or already negotiated, which is also good.

Although the drawing, as it is being drawn, is seen through each student's eyes, the primary features correlated with drawing skill development is motor control, shape formation, and color choice.  If the sheet of paper is placed over a drawing tablet, the tablet's incoming USB stream events are captured to a file, and the color selection is somehow recorded or automatically determined by having students hold the pencil or crayon up to the computer's camera before using it, a much better in natura input stream can be developed.

Pre-processing can lead to an expression of each drawing experience as a sequence of events arranged in temporal order with the following dimensions for each event.

Relative time from the instruction to draw in seconds
Color
Nearest x grid
Nearest y grid
Pressure

Determining color from camera input may be developed using LSTM approaches.

The dimensions of the label for each of these sequences would be those demographics and rankings that would most closely correlated with developmental stages.

Student age
Student gender
Curriculum grade (-1, 0, 1, 2, ... 12, where -1 is preschool and 0 is kindergarten)
Identifier of the drawing instructions given to the class
Grade ranking of the student in the class

The micro-analysis attached to each ELEMENT in the sequence includes these additional dimensions.

Drawing rate of the utensil given by $r = \frac {\sqrt{(x - x_p)^2 + (y - y_p)^2}} {t - t_p}$ where the subscript p indicates the values are drawn from the previous event in the sequence.
Drawing direction given by $\theta = \arctan (x - x_p, \; y - y_p)$
Curvature $\kappa$ calculated using cubic splines or some other data fitting approach
FFT spectrum $\vec{a}$ and Lyapunov exponent $\lambda$ applied to auto-correlation results

This is a modification of the system Google uses to synthesize speech, based on the WaveNet design. In the diagram, the residual function is defined as follows.

$z = \tanh \, (W_{f,k} x + V_{f,k} y) \, \odot \, \sigma \, (W_{g,k} x + V_{g,k} y)$

The development required is that the $\vec{a}$ must now be accompanied with scalars $r, \theta, \kappa, and \lambda$, but the resulting drawings are likely to have many of the hand-eye developmental features of the examples.

I think your task is as follows.

$\ $

Let's assume 5-year-old children.

You have numbers of pictures that drew by them.
(Let, these pictures are your [training set].)

And, you want to synthesis similar pictures with the training set.

Because you need more pictures for your study.

Am I right?

$\ $

OK.

From the pictures, you want to extract some meaningful information about a real child who drew them, right?

Then, I think GAN is not faithful for your study.

Of course, GAN can make very similar pictures with your training set.

However, it does not mean the synthesized images can contain the things what you want!

GAN just synthesizes "fake pictures" that cannot be distinguished from your training set.

The synthesized pictures may do not have any meaningful thing.

Because it is not drawn by the real child.

$\ $

But it is worth to do.

May GAN captures some features of "children-like". (But I think it is too hard.)

You can find lots of GAN for your research, especially DCGAN.


What are the latest methods to train a chat bot? I would like to train a bot that uses text input, memorizes a few categories and answers questions accordingly. In addition as version 2.0, I want to make the bot to answer voice inputs as well. Which are the latest machine learning/AI algorithms available for the same? Please let me know.

Your question is incredibly broad -- so in response, two broad frameworks I'd encourage you to look at are:

For cutting edge chatbot conversation development http://rasa.ai is an open source framework that is more adaptable than more traditional rule-based systems
For speech recognition check out https://discourse.mozilla.org/c/deep-speech that is also open source.

You can work with Recurrent Neural Nets with LSTM or GRU as memory cells and word embeddings like Word2vec.
Beam search and Attention models can also be utilized with the RNNs for more robustness and less bias.
But the outputs of these are appreciable up to some extent only as the research in this field is still hot and lot to be unraveled.


How should we regularize an LSTM model? There are five parameters from an LSTM layer for regularization if I am correct.
To deal with overfitting, I would start with

reducing the layers
reducing the hidden units
Applying dropout or regularizers.

There are kernel_regularizer, recurrent_regularizer, bias_regularizer, activity_regularizer, dropout and recurrent_dropout.
They have their definitions on the Keras's website, but can anyone share more experiences on how to reduce overfitting?
And how are these five parameters used? For example, which parameters are most frequently used and what kind of value should be input? ?

One LSTM layer should be enough unless you have lots of data. The same thing goes for the number of nodes in the layer. Start small first so 5 to 10 nodes and increment it until the performance is reasonable.
Once you have a model working you can apply regularization if you think it will improve performance by reducing overfitting of the training data. You can check this by looking at the learning curves or compring the error on the validation and test sets.
In my experiments I've used the L1 and L2 regularizers along with dropout. These can all be mixed together in fact using both L1 and L2 at the same time is called the ElasticNet.
I tend to apply the regularizers on the kernel_regularizer because this affects the weights for the inputs. Basically feature selection.
The value for the L1 and L2 can start with the default (for tensorflow) of 0.01 and change it as you see fit or read what other research papers have done.
Dropout can start at 0.1 then increment it until there is no performance gain. This is basically a percentage so 0.1 would remove about 10% of your nodes.
Finding the best regularizer is the same as any other hyperparameter optimization which is mostly trial and error.

Regularization is trying to discourage complex information from being learned so we want to eliminate the model from actually learning to memorize the training data. We don't want to learn like very specific pinpoints of the training data that don't generalize well to test data.
Dropout, the idea of drop out is that during training we randomly set some of the activations of the hidden neurons to zero with some probability say 0.5. This idea is extremely powerful because it allows the network to lower its capacity, it also makes it such that the network can't build these memorization channels through the network where it tries to just remember the data because on every iteration 50% of that data is going to be wiped out so it's going to be forced to not only generalize better but it's going to be forced to have multiple channels through the network and build a more robust representation of its prediction. We repeat this on every iteration so on the first iteration we dropped out one 50% of the nodes on the next iteration we can drop out a different randomly sampled 50% which may include some of the previously sampled nodes as well and this will allow the network to generalize better to new test data.
Early stopping, when the network is improving its performance during training there comes a point where the training data starts to diverge from the testing data, at some point the network is going to start to do better on its training data than its testing data, what this means is basically that the network is starting to memorize some of the training data and that's what you don't want so what we can do is we can identify this inflection point where the test data starts to increase and diverge from the training data so we can stop the network early and make sure that our test accuracy is as minimum as possible.


Understanding GLIE conditions for epsilon greedy approach I was going through this course on reinforcement learning (the course has two lecture videos and corresponding slides) and I had a doubt. On slide 18 of this pdf, it states following condition for an algorithm to have regret sublinear in T (T being number of pulls of multi arm bandit).

C2 - Greedy in the Limit: Let exploit(T) denote the number of pulls that are that are greedy w.r.t the empirical mean up to horizon $T$. For sub-linear regret, we need
$$\lim_{T\rightarrow\infty}\frac{\mathbb{E}[exploit(T)]}{T}=1 $$

Here, $exploit(T)$ denote the total number of "exploit" rounds performed in the first $T$ pulls. Given that expectation is defined as "the weighted sum of the outcome values, where the weights correspond to the probabilities of realizing that value",
(Q1) how exactly mathematically we define $\mathbb{E}[exploit(T)]$?
In second video (at 24:44), instructor has said that $\mathbb{E}[exploit(T)]$ is the number of exploit steps.
(Q2) Then how it equals "weighted sum of outcome values"?
(note that instructor assumes that the pulling of arm may give reward which correspond to outcome value of 1 and may not give reward which correspond to ourcome value of 0)
Also in slide 27, for GLIE-ifying $\epsilon_T$-first strategy, he selects $\epsilon_T=\frac{1}{\sqrt{T}}$. Then, the instructor counts $\sqrt{T}$ exploratory pulls and $T-\sqrt{T}$ exploitory pulls. Then to show that this satisfies condition C2, instructor states $$\mathbb{E}[exploit(T)]\geq \frac{T-\sqrt{T}}{T}$$.
Here, $\frac{T-\sqrt{T}}{T}$ is a fraction of exploitory pulls.
(Q3) So, by above equation does the instructor mean, number of exploitory pulls is greater than equal to fraction of number of exploitory pulls?
(Q4) How can we put 2nd equation in first equation and still prove limit in first equation still holds, that is, how following is the case:
$$\lim_{T=\rightarrow\infty}\frac{\frac{T-\sqrt{T}}{T}}{T}=1$$
I guess I am missing some basic concept of expectation here.

For (Q1) we mathematically define $\mathbb{E}[exploit(T)]$ as:
$$\mathbb{E}[exploit(T)] = \sum_{t=1}^T (1-\epsilon_t)$$
where $\epsilon_t$ is the exploration probability at round $t$.
The equation is basically telling us the expected number of times you pull a greedy arm over the time horizon $T$.
A more useful metric would be:
$$\mathbb{E}[explore(T)] = \sum_{t=1}^T \epsilon_t$$
It os obvious that:
$$\mathbb{E}[explore(T)] + \mathbb{E}[exploit(T)] = T$$
First, I would like to gloss over the idea why this formulation produces sublinear regret. Although there are additional assumptions that have clearly not been mentioned in the lecture, I will say it here. There is a paradigm of a 'consistent' algorithm in online learning, which in simple terms mean that the actual best arm will be identified eventually as $T \to \infty$ and by the design of your algorithm, you will be pulling this arm greedily (or with a high probability) after some time giving you a regret better than linear regret.
Now in this case, if you assume the algorithm is 'consistent' it is clear that the number of exploration steps (which is decided by exploration probability) at each step will slowly have a reduced growth as compared to the linear case i.e you are now slowly reducing exploration as your 'consistent' algorithm gurantees you have with high probability identified the best arm. From this definition/intuition given above it is clear that this means that $\mathbb{E}[explore(T)]$ should grow at a slower rate while $\mathbb{E}[exploit(T)]$ should grow at a faster rate with $T$.
From this inutitive idea rises that:
$$\lim_{T\rightarrow\infty}\frac{\mathbb{E}[exploit(T)]}{T}=1 $$
or
$$\lim_{T\rightarrow\infty}\frac{\mathbb{E}[explore(T)]}{T}=0 $$
For (Q2), it is not. It might be in expectation, but in general, it is not. I am unsure of the type of bandits used here so I will not do this calculation. In anyways, whenever sampling is involved, it means you have sampled a trajectory or a particular instance of a joint probability distribution and its expectation can never be equal to one sample (we need $\infty$ samples).(Although I am not clear on this one, the slides are not very clear to me).
For (Q3) Yes. The instructor states an inequality which I could not reproduce (I tried to do it, but it reverses the inequality sign of the instructor, you can try it yourself using the below set of equations and using the fact that $\epsilon_T \leq \epsilon_t, \quad \forall T \geq t$). Now consider the following set of equations:
$$\mathbb{E}[explore(T)] = \sum_{t=1}^T \epsilon_t$$
(a good approximate summation can be found using integrals like this)
$$\mathbb{E}[explore(T)] \approx k \sqrt T$$
where $k$ is a constant.
$$-\mathbb{E}[explore(T)] \approx -k \sqrt T$$
$$T-\mathbb{E}[explore(T)] \approx T - k \sqrt T$$
$$\frac{\mathbb{E}[exploit(T)]}{T} \approx 1-k \frac{1}{\sqrt T}$$
which answers your (Q4) that:
$$\lim_{T \to \infty} \frac{\mathbb{E}[exploit(T)]}{T} \approx \lim_{T \to \infty} 1-k \frac{1}{\sqrt T} = 1$$