Adding more Xgrid agents to your cluster might not be always as straightforward as it seems. It might not even improve the performance, or worse, decrease performance. Here are some considerations and instructions on when and how to scale your Xgrid cluster.
When should you expand your Xgrid cluster?
Simply adding an Xgrid agent doesn't make your movies being encoded faster. You have to consider parallel tasks in your workflow. For example, the 'publish to iTunes U' task will have to wait until the encoding task is finished, so these can tasks can not run in parallel. The different encoding tasks all wait for the 'intro' task to be finished and will then run in parallel. The more parallel tasks your have in a workflow, the more efficient your Xgrid cluster can be used. Another way of achieving parallel jobs is by submitting multiple Podcast Producer jobs at the same time. A single task will not run faster when you have more Xgrid agents available.
When calculating how many Xgrid agent you want to use, you will have to know how long your Podcast Producer workflows take to complete, how many jobs you will be submitting and how fast you want the material to be available after submitting. The speed of the workflow you can measure very accurately by running some tests. Make sure you test with different length recordings to get a good average. Also, you will need to maximize the CPU utilization if you want to know how many jobs a single Xgrid agent can run in a certain amount of time. To do this you will need to submit more jobs simultaneously. You could, for example, submit 20 jobs to Podcast Producer and measure the time for one Xgrid agent to finish all the jobs and divide the total time by 20 to get the time per job.
To calculate your cluster:
( T x J ) / M = A
T = time for one job to complete
J = amount of jobs simultaneously (at peak time)
M = maximum time for material to be available
A = Xgrid agents needed
Of course this is only a ball park number, but it's a starting point. If your using one XServe for measuring speed and your adding a Mac pro or iMac to the cluster, these numbers will differ.
What kind of machines should you add?
What does it matter? Adding any machine is surely better than not adding any, right? Consider the following situation: you just added a low-end Mac mini to help out with the encoding. At peak times it picks up some jobs when the first Xgrid agent is at its max capacity. Now you only submit two jobs: one short recording that is being picked up by the first Xgrid agent and one long recording that needs to be processed as fast as possible that is now being picked up by the Mac mini agent. You see the problem here? Now your important long job is stuck on the Mac mini and this could take a while. In the end it would have been faster for this job not to have the Mac mini around.
When adding machines to your cluster take into account that they are not just 'helping out' when it gets busy, they are taking on complete tasks and will not return until they are done. So make sure that the machines you add are up for the job.
There are ways of scoring machines for particular tasks, so Xgrid can decide intelligently where the jobs should go, but that is out of the scope of this article.
NFS or Xsan?
All the Xgrid agents in a cluster will need to access the same storage to be able to run your jobs. Podcast Producer 2 has made it very easy by automatically adding an NFS share for the Podcast Producer shared volume. Over a dedicated Gigabit ethernet this is fast, but only unlimitedly scalable. Your Podcast Producer server's (or NFS server's) network interface is the bottleneck here. After 6 or 7 agents (depending on your network) performance will not go up anymore.
The costs of implementing an Xsan storage will be considerably higher, but it brings great benefits. Access to the storage will be a lot faster, with standard two 4 Gb Fibre channel connections on a Fibre Channel card you will have very fast throughput. The storage is scalable, so you can add storage to the volume, without migrating all the data. Adding Xgrid (Xsan) agents will not decrease the performance like with NFS. And last, but not least, with Xsan you can configure your Podcast Producer and Xgrid services to be redundant, so you will not have a single point of failure here. Adding Xsan to your Podcast Producer environment has great benefits, but will also make system maintenance more complex. Also take into account that when you want to use Xsan, you are limited to using XServe or Mac pro machine, because these are the only Mac machines that take the necessary Fibre Channel cards.
Please check out http://www.xsanity.com/ for more information and support on Xsan.
Adding Xgrid agents
Once you have decided how, what and how many agents you want to add, we can start adding them to the cluster. Make sure that all software that you might need in your workflows is installed on the agent and that the agent has the same system version as your Xgrid controller/Podcast Producer server. You can use both Mac OS X Server and client as long as you're not using any server-specific software in your workflows. All the binaries used in a standard Podcast Composer workflow are available on both Server and Client.
If the agent you want to add is a server the steps are fairly simple. First bind your server to the Open Directory server that the Podcast Producer server is also bound to. In the default installation this is also the Podcast Producer server. In Server Admin you go to Xgrid and click on the 'Configure Xgrid Service...' button. In the following steps make sure you select 'Join a grid' and fill in the required information. Server Admin will configure this server as an Xgrid agent and your done.
When you are connecting a Mac OS X Client you will need to do a few more steps.
• First bind the machine to the Open Directory server using System Preferences - Accounts - Login Options - Network Account Server.
• Once you have added the OD server, you will need to bind the machine. Click on the 'Edit' button in System Preferences - Accounts - Login options. Then select 'Open Directory Utility...', this will open the Directory Utility app. Double-click on 'LDAPv3', select your OD server in the list and click 'Edit'. Now click on the 'Bind' button and fill in your Open Directory details.
• Once the computer is bound to OD you can go to System Preferences - Sharing and select 'Xgrid sharing'. Click on 'Configure' and select your controller. Make sure the Authenticated Method is set to 'Single Sign on'. Without single sign on, your xgrid might work, but Podcast Producer won't be able to run jobs on the agent.
• Start Xgrid sharing by ticking the box.
On the Xgrid controller you should now see the new Xgrid agent in Xgrid Admin.