This PRJ is designed to test the major render time hits like reflection, refraction, multiple ray-traced lights, shadows, anti-aliasing and combiner materials without touching on disk and RAM hogs like bitmaps or hair.

Please download this PRJ

ThreeTeapotsBenchmarkv005.prj

select the camera view in the chor.

If you are testing v16 or later, load this render preset when you render to file.

benchmark.pre

If you are testing v13-15 set your Options>Render Settings to "use settings from Camera" before you render to file. The PRJ has every possible render parameter set in the chor camera.

In either case you will need to set a file path.

Then hit render and wait. You should get a small jpg (320x240) exactly like this in your "My Documents" folder (Mac Users may have to select a new location):

Report your:

version of A:M (see Help>About A:M)

render time min:sec

CPU Brand and model

Actual CPU speed in GHz

how many cores A:M is using

RAM

OS

mine was:

15h

19:59

AMD Athlon XP 3200

1.921 GHz

1 core

2 GB RAM

Windows 2000

Thank you!

Any benchmark undertaken should help to advance an A:M User toward the creation of their image, short film, series or feature.

Any benchmark in the 1.0 series should be A:M-centric. (Comparing A:M to other programs is not the goal but comparing A:M to itself on various systems will reveal useful information that can be adjusted and optimized).

The fewer variable in a benchmark the better. With these considerations in mind we can move toward a proposal for the first 'official' A:M benchmark. Note that this would be the second series of benchmarks because the first would use only default settings found in A:M as determined by selecting "Reset Settings" in the A:M Help menu then performing and measuring a stated task.

The default benchmarks (right out of the box) would likely be termed Benchmark 0.0. Through effective benchmarking that progresses the A:M User toward their goal all variables can be known, their effects measured and altered to suit the needs of the user. Any subsequent benchmarks beyond the default should be meticulously documented with all variable identified to the maximum extent possible (an unknown variation may still be within control limits but effectively out of the user's control).

Where possible a default file should be shared with the those who will conduct the benchmark and for testing purposes it would replace the default in A:M (example: replacing the default.cho file).

Assets required for a particular benchmark can be narrowed in scope to a Project file, Choreography, Model, Action, Material, etc. to benchmark each of those areas. It is hoped the best benchmarks would then be incorporated into A:M as optimized procedures which when run automatically (at installation or other time specified by user) establish a realistic foundation from which to project the user's production schedule.

Note: Wherever possible the introduction of variables within the Benchmark should be used as an opportunity to learn. (Example: Telling A:M what file format to render to, where on the user's harddrive or network to render to and then padding the filename with zeros in order to optimize the sequential order of images produced) These represent three user specified variables that will have an effect on the user's production schedule.

A proposal for the first 'useful benchmark' beyond default settings tests would be to render a two minute movie (from a project file provided). Assets rendered from this benchmark would be used to assist in the creation of the user's own two minute movie. (This setup can then be easily altered to target still image (single frame), 30 minute short, 90 minute movie, etc.)

One thing I find hard in scoping this out is that it's difficult to find up to date benchmarks of CPUs that test single core performance.

That would be useful for judging Netrender usefulness since each Netrender node is a single-threaded process.

At the consumer level it seems like the best AMD cores take about 50% longer to do something than the best Intel cores. Hard to find comparisons of server-lever CPUs like Xeon and Opterons that can work on multi-CPU motherboards.

Anyone want to spec out a box for 2013?

Lets say under $1000 for a box (or boxes) that can run Netrender nodes and be networked with your regular A:M computer. Overclocking is fair if that is easy for the user to do.

The original scene takes 17:10 on my Athlon 3200 XP:

That version had reflections set to 8 possible levels. If I cut that to 2 we get a time of 15:31...

That's only a slight improvement because it only speeds up pixels that would have shown a reflection of more than 2 bounces. Notice the difference in the interior of the right teapot.

Those two previous scenes had three "Sun" lights using 25 rays apiece to cast their shadows. If I cut that to 5 rays I get a render time of 3:55...

It's hard to see in this small image size but the shadows are visibly grainier. But obviously the lighting was the major slow down in the first two scenes, rather than reflections or the refraction of the clear teapot. The biggest surprise is that anti-aliasing is also much briefer. I would have thought edges were edges but apparently lighting makes a difference.

If I was OK with razor sharp shadows I could cut the rays to 1 and get a time of 1:09...

If I change the lights from Sun to Klieg lights with Z-buffered shadows (not raytraced) the time goes down to 41 seconds

If I replace the mirrored Teapot with an environment mapped Teapot and turn off reflections I get 29 seconds...

That's a crude reflection map, a better one could be made with more time and for an animation of many frames it might be a benefit. But that is about the same lighting resources that Pixar has used for most of its movies. According to Pixar no ray traced shadowing, reflection or refraction was used prior to Cars. All the reflections you saw in Buzz Lightyear's visor were "fake" environment mapped reflections. Carefully done by fine lighting artists, of course.

So with A:M you have in your hand as good or better rendering capabilities than in the movies you have seen and loved, and if you use them wisely you can get economical results for your movie and not spend 17 hours per frame as Pixar did for Cars.