Podcast Producer Community Site

Welcome to podcastproducer.org
Thursday, February 09 2012 @ 08:40 AM CET

Optimizing your workflow for speed (Part 1)

Snow Leopard gives a couple of really nice compositions we can use with our recordings. All these compositions are available when we use Podcast Composer to make workflows. What you maybe don't realize is that using some of these effect might have severe effect on the performance of your workflow. The main culprit is the 'core-edit-master' task.

The core-edit-master task in a Podcast Composer workflow renders out the output of the previous effects and composition tasks. These tasks output a quicktime reference file and core-edit-master renders this file into a new flat movie file. It keeps the original size and uses H.264 (highest quality, single pass) for video and AAC (44,1 kHz, 16 bit stereo) for audio encoding. I have seen that this task in different workflows and on different hardware takes relatively the longest time of all the tasks in the workflow. There is a lot to gain in optimizing on this bit!

Here are some results from rendering times on different machines using the default Single Source workflow. All these tests were done with the same source file that I used from:

Apple Server Quicktour Movie

Time complete workflow Time edit-core-master Machine type
17:55 12:44 2.8 Ghz Quad Core XServe 6 GB RAM ATI Radeon X1300 64 MB
12:27 8:23 2.66 GHz Quad Core Mac pro 3 GB RAM NVIDIA GeForce-GT 120 521 MB
45:24 26:22 2.5 Ghz 2 core Mac Mini GMA950 64 MB

As you can see in the table, the XServe has more CPU power, but the Mac Pro still runs faster. The difference is in the GPU power. Even with the standard graphics card, the Mac Pro is still much better equipped in this department than the XServe.

Next I did some tests with a workflow I created that has no intro, no transitions and no overlay. I used the 'Single Source' workflow as template and just removed all the extra input options. It still renders to the three different formats.

Time complete workflow Time edit-core-master Machine type
8:01 3:28 2.8 Ghz Quad Core XServe 6 GB RAM ATI Radeon X1300 64 MB
6:12 2:48 2.66 GHz Quad Core Mac pro 3 GB RAM NVIDIA GeForce-GT 120 521 MB
27:39 9:44 1.66 Ghz Core 2 Mac mini GMA950 64 MB

As you can see in the results, the render times are significantly faster, but if you look inside the workflow and inspect the template.plist file, you will see that still the 'import' and 'core-edit-master' tasks are run even though they don't do a lot at this stage. I've created a workflow where I manually removed those tasks and re-run the tests.

Time complete workflow Machine type
4:38 2.8 Ghz Quad Core XServe 6 GB RAM ATI Radeon X1300 64 MB
3:53 2.66 GHz Quad Core Mac pro 3 GB RAM NVIDIA GeForce-GT 120 521 MB
17:54 1.66 Ghz Core 2 Mac mini GMA950 64 MB

As you can see, just by removing those two tasks the image result is still the same, but it now runs at twice the speed. Since using Snow Leopard you might have noticed that XGrid no longer reports the amount of core's available, but the amount of core's available divided by two. This means that on the Mac mini Core Duo Xgrid will only run one task at the same time. Now, I must say that the XGrid team at Apple have done this for a reason. I do not know the reason, so changing the value might give you unexpected results. However, if you want to give it a try here goes:


defaults write /Library/Preferences/com.apple.xgrid.agent ProcessorCount -integer
Time complete workflow Machine type
4:33 2.8 Ghz Quad Core XServe 6 GB RAM ATI Radeon X1300 64 MB
3:19 2.66 GHz Quad Core Mac pro 3 GB RAM NVIDIA GeForce-GT 120 521 MB
23:08 1.66 Ghz Core 2 Mac mini GMA950 64 MB

As you can see in my results it didn't make much of a difference for the Quad core machines, but the Mac mini was actually slower.

To round things up I can say that if you need a workflow that is fast, or you are limited by your hardware, it would be best to not use any of the special effects that Podcast Composer provides. If you need even more speed you could go into the workflow and remove the core-edit-master and import tasks by hand.

Story Options

Optimizing your workflow for speed (Part 1) | 7 comments | Create New Account
The following comments are owned by whomever posted them. This site is not responsible for what they say.
Optimizing your workflow for speed (Part 1)
Authored by: svogt on Thursday, January 21 2010 @ 05:47 PM CET

GREAT ARTICLE! I was wondering why that step was so slow. I noticed the speed increased substantially if removed "Edited Master" and "Original" from my Podcast Composer workflow.

This bring two questions:
1. I tried to take it a step further as you did and noticed most things seem to be dependent on the Edit-Core-Master task. Did you change the dependencies?
2. If I need a watermark or intro, etc, is the edited-core-master necessary or can I get around it? I assume it is necessary.

Thanks again!
Steve

Optimizing your workflow for speed (Part 1)
Authored by: Jaime Gago on Saturday, May 08 2010 @ 03:05 AM CEST

Apple could really have included that info in the man pages for the pcastactions that are using Quartz Compositions by default. This pcastaction encode task using the master.plist encoder has proven to be mandatory when using Compressor/Qmaster for encoding when the workflows uses a quartz composition (e.g. pcastaction merge).
And no doing a pcastaction flatten instead won't work, very unfortunate.
One of the reasons is that for processing via Compressor/Qmaster not all the needed files get copied to the folder (/Library/Podcast Producer/Shared/DropBox) where compressor looks for the files to encode.

GPU bottleneck
Authored by: Pascal Rigaux on Monday, May 10 2010 @ 01:17 PM CEST

Here is some tests using new mac mini. I tested on 1 minute test file (1024x768 screencast using "Best" method) with workflow "Single Source"

2.66 GHz Quad Core XServe 24 GB RAM NVIDIA GeForce-GT 120 256 MB with ProcessorCount=1:
- total time: 4:55, master time: 3:23
2.66 GHz Quad Core XServe 24 GB RAM NVIDIA GeForce-GT 120 256 MB with ProcessorCount=8 (default):
- total time: 4:27, master time: 3:23
2.26 Ghz Core 2 Mac mini 2GB RAM NVIDIA GeForce 9400 256 MB with ProcessorCount=1 (default):
- total time: 7:56, master time: 3:55
2.26 Ghz Core 2 Mac mini 2GB RAM NVIDIA GeForce 9400 256 MB with ProcessorCount=2 :
- total time: 7:33, master time: 3:59

Same test case, but the file is submitted twice at the same time (so encoding in parallel):

2.66 GHz Quad Core XServe 24 GB RAM NVIDIA GeForce-GT 120 256 MB with ProcessorCount=8:
- total time: 7:03, master time: 5:30
2.26 Ghz Core 2 Mac mini 2GB RAM NVIDIA GeForce 9400 256 MB with ProcessorCount=2:
- total time: 12:22, master time: 5:35

So, with 2 mac mini, we could encode the 2 files in 7:33, which compares nicely with 7:03 on one xserve.

The xserve cpus do not do much most of the time.
Worse, if you have many xserves, you may end up with 2 encoding-master-task being assigned to the same xserve, whereas other xserves are idle after finishing their task (since tasks are assigned mostly randomly by xgrid controller).
A workaround is to set ProcessorCount to 1. It doesn't slow down too much. Even if it makes one cry in despair to see the poor xserve being mostly unused :'-(

GPU bottleneck
Authored by: aschuett on Saturday, October 02 2010 @ 03:23 PM CEST

I believe that you could use the art.rb file to somehow compensate for this - that is, couldn't the file detect current CPU usage, and if the usage is high, lower the art.rb score so the low usage machines are preferred?

-Aaron Schuett

GPU bottleneck
Authored by: Pascal Rigaux on Friday, October 22 2010 @ 10:21 AM CEST

art.rb score is done only once per workflow, not once before each task. So to be sure that GPU is used only once, it would be necessary to "lock" the server until edit-core-master is done. It would not be that hard, but it implies many small changes in workflows, making "Podcast Composer" unusable...

 

Optimizing your workflow for speed (Part 1)
Authored by: Mineapollo on Thursday, December 02 2010 @ 12:07 PM CET

 Hi Marcel,

Thanks for the valuable info in the article. Could you detail how exactly you removed the edit-core-master elements from the template.plist? I looked at it but there seems to be a lot of processes linked to it, as someone said earlier, and I don't want to mess the whole thing up.

Thanks

 

M

Optimizing your workflow for speed (Part 1)
Authored by: Rain on Tuesday, January 11 2011 @ 03:38 PM CET

Do you mean this Snow Leopard

http://www.tubestime.com/watch/snow-leopard-10-6-1-10-6-2-sse2-sse3-intel-amd-by-hazard

? I'm new here and try to make head and tail out of it. I think if to understand it you may creat great things.