Can anyone explain why this code leaks memory?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| let leak1 () = | |
| let thread () = | |
| async { | |
| let x = Array.create 1024 0uy | |
| return () | |
| } | |
| |> Async.Start | |
| async { | |
| while true do | |
| do thread () | |
| } | |
| |> Async.RunSynchronously | |
| let leak2 () = | |
| let a1 () = async.Return() | |
| let a2 () = | |
| async { | |
| let! complete = Async.StartChild(a1 ()) | |
| return! complete | |
| } | |
| |> Async.Start | |
| let rec loop (k: int) = | |
| if k > 0 then | |
| do a2 () | |
| loop (k - 1) | |
| loop 100000 | |
| stdout.Write("PRESS ANY KEY") // >200 Mb memory use, retained | |
| System.Console.ReadLine() | |
| |> ignore |
Looks like using Async.StartAsTask makes it complete in constant space.
As far as I can tell, it's not leaking here when run in fsi (VS 11 Preview).
ReplyDeleteThanks. I played a bit more with it, looks like I can only make it leak by allocating data in a tight loop. Injecting Async.Sleep(1) in the loop makes for correct behavior.
ReplyDeleteIf you are still reading this, can you try running `leak2` in a program? I get it to use 200+ megabytes of memory without reclaiming it. Really counter-intuitive. This time I think "sleeping" does not help, it only slows down the rate at which memory is being leaked.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteI don't get a memory leak with this.
ReplyDeletelet! complete = async { return a1 () }
There must be something wrong with Async.StartChild.
This code behave exactly the same:
ReplyDeletelet leak3 () =
for x = 0 to 10 do
Array.init 10000000 id |> ignore
stdout.Write("PRESS ANY KEY")
System.Console.ReadLine()
|> ignore
I wouldn't expect any collection unless there's allocation going on.
Indentation is messed up, nothing you can't figure out.
ReplyDeleteFor my own knowledge, would it be possible for me to host code on gist and then correctly display it in a comment?
I pulled the following from Expert F# 2.0 but I don't think it has anything to do with what you are experiencing:
ReplyDelete"You should also be aware of the potential for memory stickiness. Memory stickiness occurs when the .NET
Common Language Runtime is unable to garbage-collect memory even though objects have become
unreachable. This happens especially when long-running computations and inactive callbacks hold on to
object handles related to the earlier phases of execution of a program."
@David, you can post links to GIST in comments but I can also read it without indentation.
ReplyDeleteThanks a lot for looking this up.
I have experimented a lot along the same lines. Apparently yes, some scenarios make CLR retain memory for no obvious reason. Annoying.. At least leak3 does not seem to grow linearly on repeated invocations, so it reuses the memory.
I was also experimenting with having a worker thread and a queue of tasks, and I got bad results somehow when the queue of tasks got overwhelmed with millions of them. A memory profiler I used claimed most of the memory used by the program after the thread has run is "holes", indicating fragmentation. The only way I found to make it satisfactory is putting a limit on the queue.
Concerning StartChild though, the memory profiler is showing cancellation-related objects being live in large quantities (proportional to the number of invocations). I think @gradbot has correctly identified a problem with not disposing CancellationTokenSource. Some pinned/unmanaged stuff holding on.