Talk:Particle swarm optimization

message: Hi,
I made some fairly major changes to the page with respect to how the algorithm is described and what notation is used; it's more in harmony with what's used in the literature and the books available on PSO, as far as I know.
Hi, 
Page looks great, very readable!  Good work.
I'm wondering if these formulas are backwards though:
*** If f(\mathbf{x}_i) < f(\hat{\mathbf{x}}_i), \hat{\mathbf{x}}_i \leftarrow \mathbf{x}_i.
*** If f(\mathbf{x}_i) < f(\hat{\mathbf{g}}), \hat{\mathbf{g}} \leftarrow \mathbf{x}_i.
Shouldn't it be:
*** If f(\mathbf{x}_i) > f(\hat{\mathbf{x}}_i), \hat{\mathbf{x}}_i \leftarrow \mathbf{x}_i.
*** If f(\mathbf{x}_i) > f(\hat{\mathbf{g}}), \hat{\mathbf{g}} \leftarrow \mathbf{x}_i.
:The formulas on the page are correct; in its standard form PSO is a minimizing optimizer. Of course it only takes minor changes to convert the algorithm to a maximizer. Dbratton 11:58, 15 November 2006 (UTC)
thread_title: Changes

message: how is it that J is being referenced outside the dimensions loop on the second to last line?
  for I = 1 to number of particles n do
    for J=1 to number of dimensions m do
      R1=uniform random number
      R2=uniform random number
      V[I][J]=w*V[I][J]
             +C1*R1*(P[I][J]-X[I][J])
             +C2*R2*(G[I][J]-X[I][J])
    enddo
    X[I][J] = X[I][J]+V[I][J]
  enddo
i know where this pseudocode comes from, it is from Daniel Wilke's paper. the problem is that i am not %100 sure that i have spotted an error because i have never actually seen a clear & typo-less listing of this algorithm. it's such a shame because it should be so simple and i am sure that the lack of clarity hinders adoption.
:Fixed. It was just a misplaced line. DanielC/T+ 10:47, 25 March 2007 (UTC)
thread_title:  Pseudocode typo 

message: *Particle Swarm Central is a repository for information on PSO.
: This is a decent but incomplete repository which is maintained by one researcher. It seems reasonable to me to include this link but it might give a biased presentation if it is standing on its own. Optimering (talk) 12:59, 7 October 2010 (UTC)
replies: text: Agree. —Ruud 15:22, 14 December 2010 (UTC)
thread_title: External links

message: *SwarmOps source-code library for C and C#
: This seems to be the most comprehensive implementation available for C#, which is becoming increasingly popular. It is apparently the only implementation featuring 'tuning' of the PSO parameters. It also has a parallel version of PSO. License is the standard GNU Lesser General Public License allowing for both non-commercial and commercial use. It seems reasonable to me to include this link. Optimering (talk) 12:59, 7 October 2010 (UTC)
replies: text: Agree. —Ruud 15:22, 14 December 2010 (UTC)
replies: text: Sounds like it meets #4 as a "link to consider". --Pnm (talk) 00:17, 15 December 2010 (UTC)

text: This is a comprehensive implementation in Java, which is an important cross-platform object-oriented programming language. The library appears to be of high quality and has many other features incl. multi-objective optimization. License is the standard GNU Lesser General Public License allowing for both non-commercial and commercial use. It seems reasonable to me to include this link. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Agree, academic. —Ruud 15:22, 14 December 2010 (UTC)
replies: text:  Can its comprehensiveness be verified by a reliable secondary source? If so, I think it meets #3 of "what should be linked". (The secondary source should be cited with the description.) --Pnm (talk) 00:17, 15 December 2010 (UTC)

text: Like EvA2 above but for C++ instead. This library has many features and seems to be of high quality. I am unfamiliar with this license and whether it allows for commercial use, but non-commercial use is allowed. It seems reasonable to me to include this link. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Agree, academic. —Ruud 15:22, 14 December 2010 (UTC)
replies: text:  Can the feature set and/or quality be verified by a reliable secondary source? --Pnm (talk) 00:17, 15 December 2010 (UTC)

text: This is a recent implementation of PSO variants in Matlab. There are a few such libraries in existence, most of them are considerably older. I don't feel comfortable making a judgment call which one to include but we certainly ought to link to Matlab source-code. I am also not sure what the license for this particular library (dis)allows. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Probably. —Ruud 15:22, 14 December 2010 (UTC)
replies: text:  Is there a reliable secondary source that mentions it? --Pnm (talk) 00:17, 15 December 2010 (UTC)

text: An implementation in Python would be a good idea, whether this is the best source-code available I do not know. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Generally not a fan of adding links to implementations which are neither notable nor high-quality and well-maintained. Not in favor, not against. —Ruud 15:22, 14 December 2010 (UTC)

text: This seems to be inferior to SwarmOps above and hence redundant. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Generally not a fan of adding links to implementations which are neither notable nor high-quality and well-maintained. Not in favor, not against. —Ruud 15:22, 14 December 2010 (UTC)

text: An implementation in Perl would be a good idea, whether this is the best source-code available I do not know. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Generally not a fan of adding links to implementations which are neither notable nor high-quality and well-maintained. Not in favor, not against. —Ruud 15:22, 14 December 2010 (UTC)

text: I am not familiar with Lua and have no opinion either for or against. Optimering (talk) 12:59, 7 October 2010 (UTC)

text: Generally not a fan of adding links to implementations which are neither notable nor high-quality and well-maintained. Not in favor, not against. —Ruud 15:22, 14 December 2010 (UTC)
thread_title:  Source-code 

message: *Java Applet for 3D-visualisation of PSO.
: In my opinion this is not a great visualization. It doesn't really aid in the understanding of PSO. It seems reasonable to me to exclude this link. I have searched YouTube for videos of PSO but haven't been able to find any good ones. Optimering (talk) 12:59, 7 October 2010 (UTC)
replies: text: I cannot get this to work properly even after installing the 3d-package for Java. Optimering (talk) 12:59, 7 October 2010 (UTC)
thread_title: Visualization

message: The PSO algorithm given in this article illustrates the main concepts of PSO in a simple manner. There exists a great number of PSO variants and there is disagreement amongst researchers as to what constitutes The Basic PSO. For instance, the current algorithm has for velocity update:
* vi ← ω vi + φp rp (pi-xi) + φg rg (g-xi)
where rp and  rg are scalars. Another basic PSO variant uses vectors of random numbers rp and rg instead:
* vi ← ω vi + φp rp × (pi-xi) + φg rg × (g-xi), where the × operator indicates element-by-element multiplication, that is, if the velocity is denoted vi = [vi1, ..., vin] then it is updated by computing: vij ← ω vij + φp rpj (pij-xij) + φg rgj (gj-xij), for each dimension j = 1, ..., n.
This is more difficult to understand for a novice and the former variant is therefore used. 
Similar decisions have been made for other areas of the PSO algorithm shown, such as the use of a global neighbourhood, that is, using g instead of a subset of the swarm as the neighbourhood attractor.
Optimering (talk) 05:45, 3 October 2010 (UTC)
replies: text: My issue with the algorithm as currently presented is that is not a minimization solution as described in the paragraph above.  It is indeed a maximization algorithm.  So I suggest we flip the inequalities? Sparky the one (talk) 06:34, 8 February 2023 (UTC)
thread_title:  Choice of PSO Algorithm 

message: As my mother tongue is not English, I do not want to modify the article myself, but here are some remarks that should be taken into account.
1) if someone tries to code the so called "basic PSO" as described here, he will find that the resulting algorithm is very bad. This is because the update velocity formula is wrong. In the article, we have
vi ← ω vi + φp rp (pi-xi) + φg rg (g-xi)
Mathematically, it means that all components of say (pi-xi) are multiplied by the same rp. But in fact, in the real basic PSO (1995), each component is multiplied by a different random number, to ensure a large probabilistic distribution of the next possible positions (DNPP)
In the Variant part, the article says
"pick random numbers rp and rg for each dimension when updating the velocity-vector rather than using the same numbers for the entire vector,"
This is not a all a variant, on the contrary, this is the way PSO is described and coded from the very beginning. What _is_ a variant (and a bad one), is "using the same numbers for the entire vector". It should not be presented as the basic PSO.
Here the support of the DNPP is a n-rectangle, but working "dimension by dimension" induces a well known bias: the solution point is easier found if it lies on an axis or a diagonal, and even far easier on the center of the search space. That is precisely why for the very last Standard PSO (2011) available on the Particle Swarm Central the velocity update equation is modified, by using a distribution whose support is a hypersphere. 
2) The PSO that is described here is makes use of the "global best" topology, as in the original 1995. It is bad for multimodal problems, and James Kennedy himself has written later:
"and it might be time to mount a sword-swinging crusade against any use of the gbest topology. How did this happen? It is the worst way to do it, completely unnecessary. I will not tolerate anybody who uses that topology complaining about "premature convergence.""
In the Variant section, it is said
"use a subset of the swarm (known as a neighbourhood) instead of g when updating a particle's velocity"
As it is a very important feature of PSO for more than ten years, it seems to me that a whole section should be devoted to the local best topologies, which could be divided into two classes:
*fixed (ring, von Neumann, etc.)
*variable (random, stochastic star, etc.)
It may we worth noting that the successive Standard PSOs  available on the Particle Swarm Central make use of a variable topology since 2006. 
3) A few minor points
*I do not know for native English speakers, but for me "school of though" sounds very pompous.
*"a metaheuristic such as PSO cannot be proven correct". This is meaningless, or even wrong. According to the given internal links:
**"correctness of an algorithm is asserted when it is said that the algorithm is correct with respect to a specification. "
**"a formal specification is a mathematical description of software or hardware that may be used to develop an implementation."
The specification of PSO is given by its mathematical description (mainly: initialisation, velocity update equation, moving equation, topology). Therefore, if there is no coding error (which could be automatically proved), the algorithm is correct. 
It looks like the article makes a confusion between "correctness" and "sure convergence to the optimum". Note that for some variants this "global convergence" property has the probability one. For example, you may draw the random coefficients from a distribution with an infinite support (Gaussian, Lévy, etc.). However, it is usually not an improvement (after all, even the pure random search has this property).
*"it was strongly biased in its optimization search towards similar values for different dimensions". A few words about the non-biased benchmark problems (shifted, rotated) may be useful.
*Nothing is said about the binary PSO variants, and, more generally, about discrete PSOs, and combinatorial PSOs.  
*"Particle Filter" (in the "See also" section). I am not sure that this link is very pertinent.  — Preceding unsigned comment added by MClerc (talk • contribs) 08:44, 28 December 2011 (UTC)
replies: text: * Since this is a specialist area, and your English seems quite competent, I suggest it would be better if you are bold and make the changes you have in mind to the article yourself. I doubt any other Wikipedian is currently in a better position to do this than you are. Deficiencies in your English might be a lot easier for other editors to adjust than deficiencies in their understanding of the technical subtleties. If there is any need to to correct your English, and editors are not quite sure what you meant to say, you can always be asked for clarification on the talk page. However, at the risk of trying to teach you to suck eggs, please refer to wp:Ten Simple Rules for Editing Wikipedia, wp:verification and wp:reliable sources, and make sure you wp:cite your changes appropriately. --Epipelagic (talk) 09:29, 28 December 2011 (UTC)
thread_title:  Some things that should be modified 

message: This article, though extensive, lacks any applications or examples and it desperately needs them.  By this I mean that it needs, first, general examples of problem areas that it has been applied to.  And secondly, at least one or two explicit examples of how to apply it to a problem area where it is shown how to map the algorithm's space-vectors, cost function, etc. to the problem domain and then how that results in new candidate solutions that can be mapped back to the problem's domain.
Lacking this is is very difficult for intelligent but un-initiated readers like myself to figure out how the movement of birds and fishes can be related to ..., well, to whatever kinds of practical problems PSO can be applied to.
RBarryYoung (talk) 14:27, 13 October 2012 (UTC)
thread_title:  Needed: Applications and Examples 

message: I was WP:BOLD and created a MOPSO page with a redirect to this page. I have only a vague idea of what MOPSO is and I'm not sure about the connection, but if there is one, I think the redirect could help. Any further details or corrections would be welcome. --Mlewan (talk) 17:14, 5 October 2015 (UTC)
thread_title:  MOPSO redirect