Previous Topic (None) Up (Pulse) Next Topic (PulseFAQ)

Pulse Power Management

Power management is a system for controlling how machines startup and shutdown automatically based on sets of conditions on the render farm, including job load and temperature.

Machines shutdown after they have idled for a set amount of time, and startup depending on how many tasks are queued. Machines may also poll an external temperature sensor using SNMP and shutdown a number of machines if the temperature is raised beyond a given threshold. If you have problematic machines that you find you are always restarting, you can have them reboot automatically at defined intervals. You can even set up a schedule for when slaves should start and stop.

Power management is built into Pulse, so Pulse must be running to use this feature. The only exception to this rule is Temperature checking. Redundancy for Temperature checking has been built into the Slave application, so if Pulse isn't running, you're still protected if the temperature in your farm room begins to rise. If you wish to use power management without having Pulse act as a proxy between the Slaves and the Deadline Repository, simply leave the Pulse Host Name Or IP Address setting blank in the Deadline Pulse Settings,

Power management can be configured from the Deadline Monitor while in Super User mode by selecting Tools -> Configure Power Management. The configuration process is split up into five sections:

Idle Shutdown

A system for forcing slaves to shutdown after they have been idle for set periods of time. This can be used to save on energy costs when the render farm is sitting idle. Combining this feature with Wake-On-Lan will ensure that machines in the render farm are only running when they are needed.

You can split the idle time period between a daytime period and an evening period. This is useful because in most cases, you want most of your machines to stay on during the working day, and then shutdown during the evening when there are no renders left. In addition, you can also specify exceptions to these two periods, which means (for example) you could have different idle periods for the weekend.

Idle Shutdown Settings:

  • Idle Shutdown Mode: Select Disabled, Enabled, or Debug mode. In Debug mode, all the checks are performed as normal, but no action is actually taken.
  • Slaves Will Be Shutdown After Being Idle For # Minutes: Self explanatory.
  • Number of Slaves To Leave Running: This amount of slaves to be left running.
  • Overrides: Define overrides for different days and times. Simply specify the day(s) of the week, the time period, the minimum number of slaves, and the idle shutdown time for each override required. For example if more machines are required to be running continuously for Friday evening and Saturday afternoon, it can be specified in an override.
  • Enable/Disable For These Slaves: Select which slaves the idle shutdown system will be enabled/disabled for.

Wake On Lan

A system that allows powered down machines to be started automatically when new jobs are submitted to the render farm. Combining this feature with Idle Shutdown will ensure that machines in the render farm are only running when they are needed. Only machines that support Wake On Lan (WOL) can be started up after they are shutdown.

Note: If machines begin to be shutdown due to temperature, this feature is automatically disabled to prevent machines from starting up and raising the temperature again.

Wake On Lan Settings:

  • Wake On Lan Mode: Select Disabled, Enabled, or Debug mode. In Debug mode, all the checks are performed as normal, but no action is actually taken.
  • Number Of Slaves To Wake Up Per Interval: The maximum number of machines to start in the given interval (Time Between Wake Up Intervals).
  • Disable/Enable For These Slaves: Not all machines support Wake On Lan. By enabling only the ones that do, they system will only attempt to start machines that can actually be started. All slaves will send Wake On Lan signals though, regardless of this setting.

Thermal Shutdown

The thermal shutdown system polls temperature sensors and responds by shutting down machines if the temperature gets too high. The sensors we use at Frantic Films are NetTherms, and APC Sensors are also known to be compatible.

Thermal Shutdown Settings:

  • Thermal Shutdown Mode: Select Disabled, Enabled, or Debug mode. In Debug mode, all the checks are performed as normal, but no action is actually taken.
  • Thermal Shutdown Temperature Units : The units to display and configure the temperatures in. Note that this is separate from the units that the actual sensors use.
  • Thermal Zones: You can set up different zones to monitor temperatures for. For example, we at Frantic keep track of the temperature in two areas of our server room. To add a new zone, simply click the Add button.

Zone Settings:

  • Zone Name: The name of the zone.
  • Zone Mode: If the zone is enabled or disabled.
  • Thermal Sensors: The host and OID (object identifier) of the sensor(s) in the zone. To add a sensor, simply click the Add button.
  • Temperature Thresholds: Thresholds can be added for any temperature. When a sensor reports a temperature higher than a particular threshold, the slaves in the zone will respond accordingly. Note that higher temperature thresholds take precedence over lower temperature thresholds.
  • Slaves In Zone: The slaves that will check the sensor(s) in the zone. These are also the slaves that will be shutdown when the temperature gets too high.
  • Shut down slaves if sensor(s) cannot be reached: If enabled, the slaves will be shutdown after a period of time in which the temperature sensor could not be reached for temperature information.

Sensor Settings:

  • Sensor Hostname Or IP Address: The host of the temperature sensor.
  • Sensor OID: The OID (object identifier) of the temperature sensor. The default OID is for the particular type of sensor we use.
  • Sensor SNMP Community: If testing the sensor fails when Private is selected, try selecting Public.
  • Sensor Reports Temperature As: Select the units that your temperature sensor uses to report the temperature.
  • Sensor Timeout In Milliseconds: The timeout value for contacting the sensor..
  • Test Sensor: Queries the sensor for the temperature and displays it. If the temperature displayed seems incorrect, make sure you are reading in the correct units.

Machine Restart

If you have problematic machines that you need to restart on a regular basis, you can configure the Machine Restart feature of power management to restart your slave machines at specified intervals.

Machine Restart Settings:

  • Machine Restart Mode: Select Disabled, Enabled, or Debug mode. In Debug mode, all the checks are performed as normal, but no action is actually taken.
  • Machine Groups: You can set up different groups of slave machines to restart. This is useful if you want to stagger how machines are restarted. To add a new group, simply click the Add button.

Slave Scheduling

You can use the Slave Scheduling feature of power management to configure when slaves applications should be launched and shut down.

Slave Scheduling Settings:

  • Slave Scheduling Mode: Select Disabled, Enabled, or Debug mode. In Debug mode, all the checks are performed as normal, but no action is actually taken.
  • Scheduling Groups: Groups of Slaves that you want to schedule. To add new groups, simply click the Add button.