doc.go (3895B)
1 /* 2 Generic forward-only iterator that is safe and leak-free. 3 4 This package is intended to support forward-only iteration in a variety of use 5 cases while avoiding the normal errors and leaks that can happen with iterators 6 in Go. It provides mechanisms for map/select filtering, background iteration 7 through a goroutine, and error handling throughout. 8 9 The type of the iterator is interface{}, so it can store anything, at the cost 10 that you have to cast it back out when you use it. This package can be used as 11 is, or used as an example for creating your own forward-only iterators of more 12 specific types. 13 14 sum := 0 15 iterator.Each(func(item interface{}) { 16 sum = sum + item.(int) 17 }) 18 19 Motivation 20 21 With the lack of generics and a builtin iterator pattern, iterators have been 22 the topic of much discussion in Go. Here are the discussions that inspired this: 23 24 http://ewencp.org/blog/golang-iterators/: Ewan Cheslack-Postava's 25 discussion of the major iteration patterns. Herein we have chosen the closure 26 pattern for iterator implementation, and given the choice of callback and 27 channel patterns for iteration callers. 28 29 http://blog.golang.org/pipelines: A March 2014 discussion of pipelines 30 on the go blog presents some of the pitfalls of channel iteration, suggesting 31 the "done" channel implementation to compensate. 32 33 Creating Iterators 34 35 Simple error- and cleanup-free iterators can be easily created: 36 37 // Create a simple iterator from a function 38 val := 1 39 iterator := iter.NewSimple(func() interface{} { 40 val = val * 2; 41 return val > 2000 ? val : nil // nil terminates iteration 42 }) 43 44 Typically you will create iterators in packages ("please iterate over this 45 complicated thing"). You will often handle errors and have cleanup to do. 46 iter supports both of these. You can create a fully-functional iterator thusly: 47 48 // Create a normal iterator parsing a file, close when done 49 func ParseStream(reader io.ReadCloser) iter.Iterator { 50 return iter.Iterator{ 51 Next: func() (iterator{}, error) { 52 item, err := Parse() 53 if item == nil && err == nil { 54 return nil, iter.FINISHED 55 } 56 return item, err 57 }, 58 Close: func() { 59 reader.Close() 60 } 61 } 62 } 63 64 Iterating 65 66 Callback iteration looks like this and handles any bookkeeping automatically: 67 68 // Iterate over all values 69 err := iterator.Each(func(item interface{}) { 70 fmt.Println(item) 71 }) 72 73 Sometimes you need to handle errors: 74 75 // Iterate over all values, terminating if processing has a problem 76 var files []*File 77 err := iterator.EachWithError(func(item interface{}) error { 78 file, err = os.Open(item.(string)) 79 if err == nil { 80 files = append(files, file) 81 } 82 return err 83 }) 84 85 Raw iteration looks like this: 86 87 defer iterator.Close() // allow the iterator to clean itself up 88 item, err := iterator.Next() 89 for err == nil { 90 ... // do stuff with value 91 item, err = iterator.Next() 92 } 93 if err != iter.FINISHED { 94 ... // handle error 95 } 96 97 Background goroutine iteration (using channels) deserves special mention: 98 99 // Produce the values in a goroutine, cleaning up safely afterwards. 100 // This allows background iteration to continue at its own pace while we 101 // perform blocking operations in the foreground. 102 var responses []http.Response 103 err := iterator.BackgroundEach(1000, func(item interface{}) error { 104 response, err := http.Get(item.(string)) 105 if err == nil { 106 responses = append(list, response) 107 } 108 return err 109 }) 110 111 Utilities 112 113 There are several useful functions provided to work with iterators: 114 115 // Square the ints 116 squaredIterator, err := iterator.Map(func(item interface{}) interface{} { item.int() * 2 }) 117 118 // Select non-nil values 119 nonNilIterator, err := iterator.Select(func(item interface{}) bool) { item != nil }) 120 121 // Produce a list 122 list, err := iterator.ToList() 123 124 */ 125 package iter