How to use Criterion to measure performance of Haskell programs? -

How to use Criterion to measure performance of Haskell programs? -

i'm trying measure performance of simple haar dwt programme using criterion framework. (it erroneously slow, i'll leave question). can't find documentation on web, unfortunately. 2 primary problems are

how can 1 pass info 1 benchmark another? want time each stage of program. how sampling work, , avoid lazy evaluation reusing previous computations?

this source relatively pared down; first function getrandlist generates list of random numbers; haarstep transforms input signal differences , sums, , haardwt calls former , recurses on sums. i'm trying pass getrandlist haardwt via lazy evaluation, perhaps usage wrong / unsupported. timings don't seem create sense.

{-# language viewpatterns #-} import control.arrow import qualified data.vector.unboxed v import system.random import criterion.main invsqrt2 = 0.70710678118654752440 getrandlist :: randomgen g => g -> int -> [float] getrandlist gen 0 = [] getrandlist gen n = v:rest (v, gen') = random gen rest = getrandlist gen' (n - 1) haarstep :: v.vector float -> (v.vector float, v.vector float) haarstep = (alternatingop (-) &&& alternatingop (+)) alternatingop op x = v.generate (v.length x `div` 2) (\i -> ((x v.! (2 * i)) `op` (x v.! (2 * + 1))) * invsqrt2) haardwt :: v.vector float -> v.vector float haardwt xl@(v.length -> 1) = xl haardwt (haarstep -> (d, s)) = haardwt s v.++ d main = gen <- getstdgen indata <- homecoming $ getrandlist gen 2097152 outdata <- homecoming $ haardwt (v.fromlist indata) defaultmain [ bench "get input" $ nf id indata, bench "transform" $ nf v.tolist outdata ] writefile "input.dat" (unlines $ map show indata) writefile "output.dat" (unlines $ map show $ v.tolist outdata)

finally, i'm getting error when seek phone call -s 1; maybe criterion bug.

main: ./data/vector/generic.hs:237 ((!)): index out of bounds (1,1)

thanks in advance!

the posted benchmark erroniously slow... or it

are sure it's erroneous? you're touching (well, "nf" phone call touching) 2 1000000 boxed elements - thats 4 1000000 pointers. can phone call erroneous if want, issue think you're measure compared measuring.

sharing info between benchmarks

data sharing can accomplished through partial application. in benchmarks commonly have

let var = somethingcommon in defaultmain [ bench "one" (nf (func1 somethingcommon) input1) , bench "two" (nf (func2 somethingcommon) input2)]

avoiding reuse in presences of lazy evaluation

criterion avoids sharing separating out function , input. have signatures such as:

functobenchmark :: (nfdata b) => -> b inputforfunc ::

in haskell every time apply functobenchmark inputforfunc create thunk needs evaluated. there no sharing unless utilize same variable name previous computation. there no automatic memoization - seems mutual misunderstanding.

notice nuance in what isn't shared. aren't sharing final result, input shared. if generation of input want benchmark (i.e. getrandlist, in case) benchmark , not identity + nf function:

main = gen <- getstdgen allow indata = getrandlist gen size invec = v.fromlist indata size = 2097152 defaultmain [ bench "get input real" $ nf (getrandlist gen) size , bench "get input real , run harrdwt , listify vector" $ nf (v.tolist . haardwt . v.fromlist . getrandlist gen) size , bench "screw generation, how fast haardwt" $ whnf haardwt invec] -- unboxed vectors whnf sufficient

interpreting data

the 3rd benchmark rather instructive. lets @ criterion prints out:

benchmarking screw generation, how fast haardwt collecting 100 samples, 1 iterations each, in estimated 137.3525 s bootstrapping 100000 resamples mean: 134.7204 ms, lb 134.5117 ms, ub 135.0135 ms, ci 0.950

based on single run, criterion thinks take 137 seconds perform it's 100 samples. 10 seconds later done - happened? well, first run forced inputs (invec), expensive. subsequent runs found value instead of thunk, , truely benchmarked haardwt , not stdgen rng (which known painfully slow).

haskell performance criterion