View source for Kruskal's algorithm

{{short description|Minimum spanning forest algorithm that greedily adds edges}}
{{Distinguish|Kruskal's principle}}
{{Infobox Algorithm
|image=[[File:KruskalDemo.gif|frameless|upright=1.35]]
|caption=Animation of Kruskal's algorithm on a [[complete graph]] with weights based on Euclidean distance
|class=[[Minimum spanning tree|Minimum spanning tree algorithm]]
|data=[[Graph (abstract data type)|Graph]]
|time=<math>O(|E|\log |V|)</math>
}}
'''Kruskal's algorithm'''<ref>{{Cite book |last=Kleinberg |first=Jon |url=https://www.worldcat.org/oclc/57422612 |title=Algorithm design |date=2006 |publisher=Pearson/Addison-Wesley |others=Éva Tardos |isbn=0-321-29535-8 |location=Boston |pages=142–151 |oclc=57422612}}</ref> finds a [[minimum spanning tree|minimum spanning forest]] of an undirected [[weighted graph|edge-weighted graph]]. If the graph is [[Connectivity (graph theory)|connected]], it finds a [[minimum spanning tree]]. It is a [[greedy algorithm]] that in each step adds to the forest the lowest-weight edge that will not form a [[Cycle (graph theory)|cycle]].<ref name=":0">{{Cite book|last1=Cormen|first1=Thomas|url=https://archive.org/details/introductiontoal00corm_805|title=Introduction To Algorithms|last2=Charles E Leiserson, Ronald L Rivest, Clifford Stein|publisher=MIT Press|year=2009|isbn=978-0262258104|edition=Third|pages=[https://archive.org/details/introductiontoal00corm_805/page/n651 631]|url-access=limited}}</ref> The key steps of the algorithm are [[sorting]] and the use of a [[disjoint-set data structure]] to detect cycles. Its running time is dominated by the time to sort all of the graph edges by their weight.

A minimum spanning tree of a connected weighted graph is a connected subgraph, without cycles, for which the sum of the weights of all the edges in the subgraph is minimal. For a disconnected graph, a minimum spanning forest is composed of a minimum spanning tree for each [[Connected component (graph theory)|connected component]].

This algorithm was first published by [[Joseph Kruskal]] in 1956,<ref>{{Cite journal | last1 = Kruskal | first1 = J. B. | author-link1 = Joseph Kruskal| doi = 10.1090/S0002-9939-1956-0078686-7 | title = On the shortest spanning subtree of a graph and the traveling salesman problem | journal = [[Proceedings of the American Mathematical Society]] | volume = 7 | issue = 1 | pages = 48–50 | year = 1956| jstor = 2033241| doi-access = free }}</ref> and was rediscovered soon afterward by {{harvtxt|Loberman|Weinberger|1957}}.<ref>{{cite journal | last1 = Loberman | first1 = H. | last2 = Weinberger | first2 = A. | date = October 1957 | doi = 10.1145/320893.320896 | issue = 4 | journal = Journal of the ACM | pages = 428–437 | title = Formal Procedures for connecting terminals with a minimum total wire length | volume = 4| s2cid = 7320964 | doi-access = free }}</ref> Other algorithms for this problem include [[Prim's algorithm]], [[Borůvka's algorithm]], and the [[reverse-delete algorithm]].

==Algorithm==
The algorithm performs the following steps:
* Create a forest (a set of trees) initially consisting of a separate single-vertex tree for each vertex in the input graph.
* Sort the graph edges by weight.
* Loop through the edges of the graph, in ascending sorted order by their weight. For each edge:
**Test whether adding the edge to the current forest would create a cycle.
**If not, add the edge to the forest, combining two trees into a single tree.

At the termination of the algorithm, the forest forms a minimum spanning forest of the graph. If the graph is connected, the forest has a single component and forms a minimum spanning tree.

==Pseudocode==
The following code is implemented with a [[disjoint-set data structure]]. It represents the forest ''F'' as a set of undirected edges, and uses the disjoint-set data structure to efficiently determine whether two vertices are part of the same tree.

 '''algorithm''' Kruskal(''G'') '''is'''
     F:= ∅
     '''for each''' v '''in''' G.V '''do'''
         MAKE-SET(v)
     '''for each''' {u, v} '''in''' G.E ordered by weight({u, v}), increasing '''do'''
         '''if''' FIND-SET(u) ≠ FIND-SET(v) '''then'''
             F := F ∪ { {u, v} }
             UNION(FIND-SET(u), FIND-SET(v))
     '''return''' F

== Complexity ==

For a graph with {{mvar|E}} edges and {{mvar|V}} vertices, Kruskal's algorithm can be shown to run in time {{math|''O''(''E'' log ''E'')}} time, with simple data structures. Here, {{mvar|O}} expresses the time in [[big O notation]], and {{math|log}} is a [[logarithm]] to any base (since inside {{mvar|O}}-notation logarithms to all bases are equivalent, because they are the same up to a constant factor). This time bound is often written instead as {{math|''O''(''E'' log ''V'')}}, which is equivalent for graphs with no isolated vertices, because for these graphs {{math|''V''/2 ≤ ''E'' < ''V''<sup>2</sup>}} and the logarithms of {{mvar|V}} and {{mvar|E}} are again within a constant factor of each other.

To achieve this bound, first sort the edges by weight using a [[comparison sort]] in {{math|''O''(''E'' log ''E'')}} time. Once sorted, it is possible to loop through the edges in sorted order in constant time per edge. Next, use a [[disjoint-set data structure]], with a set of vertices for each component, to keep track of which vertices are in which components. Creating this structure, with a separate set for each vertex, takes {{mvar|V}} operations and {{math|''O''(''V'')}} time. The final iteration through all edges performs two find operations and possibly one union operation per edge. These operations take [[amortized time]] {{math|''O''(''α''(''V''))}} time per operation, giving worst-case total time {{math|''O''(''E'' ''α''(''V''))}} for this loop, where {{mvar|α}} is the extremely slowly growing [[inverse Ackermann function]]. This part of the time bound is much smaller than the time for the sorting step, so the total time for the algorithm can be simplified to the time for the sorting step.

In cases where the edges are already sorted, or where they have small enough integer weight to allow [[integer sorting]] algorithms such as [[counting sort]] or [[radix sort]] to sort them in linear time, the disjoint set operations are the slowest remaining part of the algorithm and the total time is {{math|''O''(''E'' ''α(''V''))}}.

== Example ==

{| class="wikitable"
! Image !! Description
|-
|[[Image:Kruskal Algorithm 1.svg|200px]]
|'''AD''' and '''CE''' are the shortest edges, with length 5, and '''AD''' has been [[Arbitrary|arbitrarily]] chosen, so it is highlighted.
|-
|[[Image:Kruskal Algorithm 2.svg|200px]]
|'''CE''' is now the shortest edge that does not form a cycle, with length 5, so it is highlighted as the second edge.
|-
|[[Image:Kruskal Algorithm 3.svg|200px]]
|The next edge, '''DF''' with length 6, is highlighted using much the same method.
|-
|[[Image:Kruskal Algorithm 4.svg|200px]]
|The next-shortest edges are '''AB''' and '''BE''', both with length 7. '''AB''' is chosen arbitrarily, and is highlighted. The edge '''BD''' has been highlighted in red, because there already exists a path (in green) between '''B''' and '''D''', so it would form a cycle ('''ABD''') if it were chosen.
|-
|[[Image:Kruskal Algorithm 5.svg|200px]]
|The process continues to highlight the next-smallest edge, '''BE''' with length 7. Many more edges are highlighted in red at this stage: '''BC''' because it would form the loop '''BCE''',  '''DE''' because it would form the loop '''DEBA''', and '''FE''' because it would form '''FEBAD'''.
|-
|[[Image:Kruskal Algorithm 6.svg|200px]]
|Finally, the process finishes with the edge '''EG''' of length 9, and the minimum spanning tree is found.
|}

== Proof of correctness ==

The proof consists of two parts. First, it is proved that the algorithm produces a [[spanning tree]]. Second, it is proved that the constructed spanning tree is of minimal weight.

===Spanning tree===
Let <math>G</math> be a connected, weighted graph and let <math>Y</math> be the subgraph of <math>G</math> produced by the algorithm. <math>Y</math> cannot have a cycle, as by definition an edge is not added if it results in a cycle. <math>Y</math> cannot be disconnected, since the first encountered edge that joins two components of <math>Y</math> would have been added by the algorithm. Thus, <math>Y</math> is a spanning tree of <math>G</math>.

===Minimality===

We show that the following proposition '''''P''''' is true by [[Mathematical induction|induction]]: If ''F'' is the set of edges chosen at any stage of the algorithm, then there is some minimum spanning tree that contains ''F'' and none of the edges rejected by the algorithm.
* Clearly '''''P''''' is true at the beginning, when ''F'' is empty: any minimum spanning tree will do, and there exists one because a weighted connected graph always has a minimum spanning tree.
* Now assume '''''P''''' is true for some non-final edge set ''F'' and let ''T'' be a minimum spanning tree that contains ''F''.
** If the next chosen edge ''e'' is also in ''T'', then '''''P''''' is true for ''F'' + ''e''.
**Otherwise, if ''e'' is not in ''T'' then ''T'' + ''e'' has a cycle ''C''. The cycle ''C'' contains edges which do not belong to  ''F'' + ''e'', since ''e'' does not form a cycle when added to ''F'' but does in ''T''.  Let ''f'' be an edge which is in ''C'' but not in ''F'' + ''e''.  Note that ''f'' also belongs to ''T'', since ''f'' belongs to ''T'' + ''e'' but not ''F'' + ''e''. By  '''''P''''', ''f'' has not been considered by the algorithm. ''f'' must therefore have a weight at least as large as ''e''. Then ''T'' &minus; ''f'' + ''e'' is a tree, and it has the same or less weight as ''T''. However since ''T'' is a minimum spanning tree then ''T'' &minus; ''f'' + ''e'' has the same weight as ''T'', otherwise we get a contradiction and ''T'' would not be a minimum spanning tree. So ''T'' &minus; ''f'' + ''e'' is a minimum spanning tree containing ''F'' + ''e'' and again '''''P''''' holds.
* Therefore, by the principle of induction, '''''P''''' holds when ''F'' has become a spanning tree, which is only possible if ''F'' is a minimum spanning tree itself.

== Parallel algorithm ==

Kruskal's algorithm is inherently sequential and hard to parallelize. It is, however, possible to perform the initial sorting of the edges in parallel or, alternatively, to use a parallel implementation of a [[binary heap]] to extract the minimum-weight edge in every iteration.<ref>{{Cite journal|last1=Quinn|first1=Michael J.|last2=Deo|first2=Narsingh|date=1984|title=Parallel graph algorithms|journal=ACM Computing Surveys |volume= 16 |issue=3 |pages=319–348|doi=10.1145/2514.2515|s2cid=6833839|doi-access=free}}</ref>
As parallel sorting is possible in time <math>O(n)</math> on <math>O(\log n)</math> processors,<ref>{{cite book|title=Introduction to Parallel Computing|last1=Grama|first1=Ananth|last2=Gupta|first2=Anshul|last3=Karypis|first3=George|last4=Kumar|first4=Vipin|year=2003|isbn=978-0201648652|pages=412–413}}</ref> the runtime of Kruskal's algorithm can be reduced to ''O''(''E'' α(''V'')), where α again is the inverse of the single-valued [[Ackermann function]].

A variant of Kruskal's algorithm, named Filter-Kruskal, has been described by Osipov et al.<ref name="osipov2009">{{Cite journal|last1=Osipov|first1=Vitaly|last2=Sanders|first2=Peter|last3=Singler|first3=Johannes|date=2009|title=The filter-kruskal minimum spanning tree algorithm|url=http://algo2.iti.kit.edu/documents/fkruskal.pdf|journal=Proceedings of the Eleventh Workshop on Algorithm Engineering and Experiments (ALENEX). Society for Industrial and Applied Mathematics|pages=52–61}}</ref> and is better suited for parallelization. The basic idea behind Filter-Kruskal is to partition the edges in a similar way to [[quicksort]] and filter out edges that connect vertices of the same tree to reduce the cost of sorting. The following [[pseudocode]] demonstrates this.
 
 '''function''' filter_kruskal(G) '''is'''
     '''if''' |G.E| < kruskal_threshold:
         '''return''' kruskal(G)
     pivot = choose_random(G.E)
     E{{sub|≤}}, E{{sub|>}} = partition(G.E, pivot)
     A = filter_kruskal(E{{sub|≤}})
     E{{sub|>}} = filter(E{{sub|>}})
     A = A ∪ filter_kruskal(E{{sub|>}})
     '''return''' A
 
 '''function''' partition(E, pivot) '''is'''
     E{{sub|≤}} = ∅, E{{sub|>}} = ∅
     '''foreach''' (u, v) in E '''do'''
         '''if''' weight(u, v) ≤ pivot '''then'''
             E{{sub|≤}} = E{{sub|≤}} ∪ {(u, v)}
         '''else'''
             E{{sub|>}} = E{{sub|>}} ∪ {(u, v)}
     '''return''' E{{sub|≤}}, E{{sub|>}}
 
 '''function''' filter(E) '''is'''
     E{{sub|f}} = ∅
     '''foreach''' (u, v) in E '''do'''
         '''if''' find_set(u) ≠ find_set(v) '''then'''
             E{{sub|f}} = E{{sub|f}} ∪ {(u, v)}
     '''return''' E{{sub|f}}

Filter-Kruskal lends itself better to parallelization as sorting, filtering, and partitioning can easily be performed in parallel by distributing the edges between the processors.<ref name="osipov2009" />

Finally, other variants of a parallel implementation of Kruskal's algorithm have been explored. Examples include a scheme that uses helper threads to remove edges that are definitely not part of the MST in the background,<ref>{{Cite book|last1=Katsigiannis|first1=Anastasios|last2=Anastopoulos|first2=Nikos|last3=Konstantinos|first3=Nikas|last4=Koziris|first4=Nectarios|title=2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum |chapter=An Approach to Parallelize Kruskal's Algorithm Using Helper Threads |date=2012|url=http://tarjomefa.com/wp-content/uploads/2017/10/7793-English-TarjomeFa.pdf|pages=1601–1610|doi=10.1109/IPDPSW.2012.201|isbn=978-1-4673-0974-5|s2cid=14430930}}</ref> and a variant which runs the sequential algorithm on ''p'' subgraphs, then merges those subgraphs until only one, the final MST, remains.<ref>{{Cite journal|last1=Lončar|first1=Vladimir|last2=Škrbić|first2=Srdjan|last3=Balaž|first3=Antun|date=2014|title=Parallelization of Minimum Spanning Tree Algorithms Using Distributed Memory Architectures|url=https://www.researchgate.net/publication/235994104|journal=Transactions on Engineering Technologies.|pages=543–554|doi=10.1007/978-94-017-8832-8_39|isbn=978-94-017-8831-1}}</ref>

==See also==
* [[Prim's algorithm]]
* [[Dijkstra's algorithm]]
* [[Borůvka's algorithm]]
* [[Reverse-delete algorithm]]
* [[Single-linkage clustering]]
* [[Greedy geometric spanner]]

== References ==
{{Reflist}}
* [[Thomas H. Cormen]], [[Charles E. Leiserson]], [[Ronald L. Rivest]], and [[Clifford Stein]]. ''[[Introduction to Algorithms]]'', Second Edition. MIT Press and McGraw-Hill, 2001. {{isbn|0-262-03293-7}}. Section 23.2: The algorithms of Kruskal and Prim, pp.&nbsp;567&ndash;574.
* [[Michael T. Goodrich]] and [[Roberto Tamassia]]. ''Data Structures and Algorithms in Java'', Fourth Edition. John Wiley & Sons, Inc., 2006. {{isbn|0-471-73884-0}}. Section 13.7.1: Kruskal's Algorithm, pp.&nbsp;632..

== External links ==
* [https://github.com/carlschroedl/kruskals-minimum-spanning-tree-algorithm-example-data Data for the article's example].
* [https://gephi.org/plugins/#/plugin/spanning-tree-plugin Gephi Plugin For Calculating a Minimum Spanning Tree] [https://github.com/carlschroedl/gephi-plugins/tree/minimum-spanning-tree-plugin/modules/MinimumSpanningTree source code].
* [http://www.geeksforgeeks.org/kruskals-minimum-spanning-tree-using-stl-in-c/ Kruskal's Algorithm with example and program in c++]
* [https://meyavuz.wordpress.com/2017/03/11/how-does-kruskals-algorithm-progress/ Kruskal's Algorithm code in C++ as applied to random numbers]
* [https://gist.github.com/DanilAndreev/e519d77eff91f03f09616c9170db7941 Kruskal's Algorithm code in Python with explanation]

{{Graph traversal algorithms}}

[[Category:Graph algorithms]]
[[Category:Spanning tree]]
[[Category:Articles with example pseudocode]]
[[Category:Articles containing proofs]]
[[Category:Greedy algorithms]]