Deadline 的 Web Service 自訂 Script 跟 REST心得

2018/8/28 上午6:48:40

最近練習用 react 做了一個讓同事可以直接用網頁或手機看算圖的網站。

從超級難用的 deadline documentation 總結出一些心得。

Web Service

雖然 deadline 有提供 REST api,但功能有限,譬如取得 jobs 篩選的方式就只能靠 status,沒辦法找特定 user 的,而且取得的 job 資料量非常龐大,包含外掛參數什麼都有,讓來往的封包沒意義的龐大。
所幸 Web Service 有 Web Service Scripts 這個功能,讓使用者可以用 Python 自訂資訊,足以彌補這方面的缺陷。

Web Service Scripts 的資料夾就在 DeadlineRepository\scripts\WebService 底下,裡面已經有一些內建的 script 是給 deadline 本身自有的 APP 用的。
只要將 demo.py 放在裡面,就可以用 http://host:port/demo 執行該請求,詳細的方法都在上面連結文件中,這邊就不多述。

Script 範例

雖然資料夾下就有一些現成 script,但那些文件因為架構很嚴謹難以去解讀,所以使用我做上面那個網站所寫的其中一個 script。
這個 script 檔名是 jobs.py,只要使用者連上 http://host:port/jobs,就可以取得目前正在算圖或排隊中的 jobs。
那如果有指定使用者,譬如 http://host:port/jobs?username=elisha,就會取得該使用者送出的 jobs。

from Deadline.Scripting import * import json def __main__(dlArgs, qsArgs): filter_name = qsArgs.get('username', None) # 用qsArgs取得參數 jobs = WebServiceUtils.GetJobs() # 取得全部的jobs # 開始過濾 result = [] for job in jobs: # 如果沒有指定使用者名稱,過濾非活動中的jobs if filter_name is None and job['Status'] not in ['Active', 'Pending']: continue # 如果有指定使用者,過濾非使用者的jobs if filter_name is not None and job['UserName'] != filter_name: continue # 將只有網站需要的屬性map到回應的物件 obj = { 'name': job['Name'], 'batch': job['BatchName'], 'plugin': job['PluginName'], 'user': job['UserName'], 'status': job['Status'], 'priority': job['Priority'], 'id': job['JobId'], 'submitDate': job['SubmitDateTimeString'], 'startDate': job['StartedDateTimeString'], 'completedDate': job['CompletedDateTimeString'], 'firstFrame': job['FirstFrame'], 'lastFrame': job['LastFrame'], 'taskCount': job['TaskCount'], 'pool': job['Pool'], 'completedTasks': job['JobCompletedTasks'], 'queuedTasks': job['JobQueuedTasks'], 'suspendedTasks': job['JobSuspendedTasks'], 'renderingTasks': job['JobRenderingTasks'], 'failedTasks': job['JobFailedTasks'], 'pendingTasks': job['JobPendingTasks'], 'errors': job['ErrorReports'], 'singleProgress': job['SingleTaskProgress'], } result.append(obj) return json.dumps(result) # 也可以是tuple,typle第二個數值是status code,預設為200

WebServiceUtils 算是這類 script 的核心,不同於其他難搞的 deadline 模組,WebServiceUtils.GetJobs() 會取得帶有全部資訊的 job 字典陣列,只是這些 job 不是 Python 原生的字典物件,無法直接使用 json.dumps() 給序列化,所以屬性還需個別取出。

CORS 問題

如果在網站上另外對 deadline 伺服器發送請求會產生 CORS 的問題,而 webserive 應用程式本身已經封裝起來,目前是沒看到有設定可以開啟 CORS,所以架這網站時有另外使用 CORS Anywhere 的方法來解決。

job 與 task 字典內容

WebServiceUtils 取得的 job 跟 task 內容一般情況比較難看到,順便附上。

job 物件

"job": [ "Name", "BatchName", "UserName", "RegionName", "Comment", "Department", "FramesList", "ChunkSize", "TaskCount", "Group", "Pool", "SecondaryPool", "Priority", "ConcurrentTasks", "LimitTasksToNumberOfCpus", "ReloadRenderer", "SynchronizeAllAuxiliaryFiles", "InterruptibleFlag", "InterruptiblePercentage", "SequentialJobFlag", "SuppressEvents", "EnableAutoTimeout", "EnableTimeoutsForScriptTasks", "EnableFrameTimeouts", "OnTaskTimeout", "MinRenderTime", "MinRenderTimeSeconds", "TaskTimeout", "TaskTimeoutSeconds", "StartJobTimeout", "StartJobTimeoutSeconds", "InitializePluginTimeout", "InitializePluginTimeoutSeconds", "JobDependencies", "IsFrameDependent", "FrameDependencyOffsetStart", "FrameDependencyOffsetEnd", "ResumeOnCompleteDependencies", "ResumeOnDeletedDependencies", "ResumeOnFailedDependencies", "JobDependencyPercentage", "IgnoreBadJobDetection", "OverrideAutoJobCleanup", "OverrideJobCleanup", "JobCleanupDays", "OverrideJobCleanupType", "OverrideJobFailureDetection", "FailureDetectionJobErrors", "OverrideTaskFailureDetection", "FailureDetectionTaskErrors", "OverrideNotificationMethod", "EmailNotification", "PopupNotification", "NotificationEmails", "NotificationTargets", "NotificationNote", "SendJobErrorWarning", "OnJobComplete", "Protected", "LimitGroups", "ListedSlaves", "WhitelistFlag", "MachineLimit", "MachineLimitProgress", "PreJobScript", "PostJobScript", "PreTaskScript", "PostTaskScript", "ScheduledType", "ScheduledDays", "ScheduledStartDateTime", "ScheduledStopDateTime", "MondayStartTime", "MondayStopTime", "TuesdayStartTime", "TuesdayStopTime", "WednesdayStartTime", "WednesdayStopTime", "ThursdayStartTime", "ThursdayStopTime", "FridayStartTime", "FridayStopTime", "SaturdayStartTime", "SaturdayStopTime", "SundayStartTime", "SundayStopTime", "ExtraInfo0", "ExtraInfo1", "ExtraInfo2", "ExtraInfo3", "ExtraInfo4", "ExtraInfo5", "ExtraInfo6", "ExtraInfo7", "ExtraInfo8", "ExtraInfo9", "ExtraInfoDictionary", "OverrideTaskExtraInfoNames", "TaskExtraInfoName0", "TaskExtraInfoName1", "TaskExtraInfoName2", "TaskExtraInfoName3", "TaskExtraInfoName4", "TaskExtraInfoName5", "TaskExtraInfoName6", "TaskExtraInfoName7", "TaskExtraInfoName8", "TaskExtraInfoName9", "CustomPluginDirectory", "CustomEventPluginDirectory", "RequiredAssets", "AWSPortalAssets", "ScriptDependencies", "CompletedFrames", "TotalRenderTimeTicks", "TotalRenderTime", "IsSubmitted", "IsBeingPurged", "SubmitMachineName", "SubmitUserName", "SubmitDateTimeString", "SubmitDateTime", "StartedDateTimeString", "StartedDateTime", "CompletedDateTimeString", "CompletedDateTime", "PluginName", "OutputDirectories", "OutputFileNames", "OutputTileFileNames", "MaintenanceJob", "MaintenanceJobStartFrame", "MaintenanceJobEndFrame", "TileJob", "TileJobFrame", "TileJobTileCount", "TileJobTilesInX", "TileJobTilesInY", "Status", "AuxiliarySubmissionFileNames", "BadSlaves", "InitialCompletedTaskIds", "InitialUncompletedTaskIds", "PluginInfoDictionary", "EventOptIns", "EnvironmentDictionary", "UseJobEnvironmentOnly", "ExtraInfoKeyValues", "Frames", "FirstFrame", "LastFrame", "CompletedChunks", "QueuedChunks", "SuspendedChunks", "RenderingChunks", "FailedChunks", "PendingChunks", "SingleTaskProgress", "LogReportFileNames", "LogReportLastWriteTime", "ErrorReports", "PluginDataFileName", "PluginDataFileSize", "JobName", "JobBatchName", "JobComment", "JobDepartment", "JobPool", "RemTimeThreshold", "JobSecondaryPool", "JobGroup", "JobPriority", "JobUserName", "JobMinRenderTimeSeconds", "JobTaskTimeoutSeconds", "JobStartJobTimeoutSeconds", "JobInitializePluginTimeoutSeconds", "JobOnTaskTimeout", "JobEnableAutoTimeout", "JobEnableTimeoutsForScriptTasks", "JobEnableFrameTimeouts", "JobConcurrentTasks", "JobLimitTasksToNumberOfCpus", "JobOnJobComplete", "JobProtected", "JobScheduledType", "JobScheduledDays", "JobScheduledStartDateTime", "JobScheduledStopDateTime", "JobMondayStartTime", "JobMondayStopTime", "JobTuesdayStartTime", "JobTuesdayStopTime", "JobWednesdayStartTime", "JobWednesdayStopTime", "JobThursdayStartTime", "JobThursdayStopTime", "JobFridayStartTime", "JobFridayStopTime", "JobSaturdayStartTime", "JobSaturdayStopTime", "JobSundayStartTime", "JobSundayStopTime", "JobInterruptible", "JobInterruptiblePercentage", "JobForceReloadPlugin", "JobSequentialJob", "JobSynchronizeAllAuxiliaryFiles", "JobSuppressEvents", "JobOverrideNotificationMethod", "JobEmailNotification", "JobPopupNotification", "JobNotificationNote", "JobOverrideAutoJobCleanup", "JobOverrideJobCleanup", "JobOverrideJobCleanupDays", "JobOverrideJobCleanupType", "JobSendJobErrorWarning", "JobOverrideJobFailureDetection", "JobFailureDetectionJobErrors", "JobOverrideTaskFailureDetection", "JobFailureDetectionTaskErrors", "JobIgnoreBadSlaveDetection", "JobPreJobScript", "JobPostJobScript", "JobPreTaskScript", "JobPostTaskScript", "JobIsFrameDependent", "JobFrameDependencyOffsetStart", "JobFrameDependencyOffsetEnd", "JobResumeOnCompleteDependencies", "JobResumeOnDeletedDependencies", "JobResumeOnFailedDependencies", "JobDependencyPercentageValue", "JobUseJobEnvironmentOnly", "JobCustomPluginDirectory", "JobCustomEventPluginDirectory", "JobExtraInfo0", "JobExtraInfo1", "JobExtraInfo2", "JobExtraInfo3", "JobExtraInfo4", "JobExtraInfo5", "JobExtraInfo6", "JobExtraInfo7", "JobExtraInfo8", "JobExtraInfo9", "JobOverrideTaskExtraInfoNames", "JobTaskExtraInfoName0", "JobTaskExtraInfoName1", "JobTaskExtraInfoName2", "JobTaskExtraInfoName3", "JobTaskExtraInfoName4", "JobTaskExtraInfoName5", "JobTaskExtraInfoName6", "JobTaskExtraInfoName7", "JobTaskExtraInfoName8", "JobTaskExtraInfoName9", "JobCompletedTasks", "JobQueuedTasks", "JobSuspendedTasks", "JobRenderingTasks", "JobFailedTasks", "JobPendingTasks", "JobSubmitMachine", "JobSubmitDateTime", "JobStartedDateTime", "JobCompletedDateTime", "JobId", "JobStatus", "JobMaintenanceJob", "JobMaintenanceJobStartFrame", "JobMaintenanceJobEndFrame", "JobTileJob", "JobTileJobFrame", "JobTileJobTileCount", "JobTileJobTilesInX", "JobTileJobTilesInY", "JobLimitGroups", "JobListedSlaves", "JobWhitelistFlag", "JobMachineLimit", "JobMachineLimitProgress", "JobNotificationTargets", "JobNotificationEmails", "JobAuxiliarySubmissionFileNames", "JobOutputDirectories", "JobOutputFileNames", "JobOutputTileFileNames", "JobFramesPerTask", "JobFrames", "JobFramesList", "JobPlugin", "JobTaskCount", "JobDependencyIDs", "JobRequiredAssets", "JobScriptDependencies", "LastWriteTime", "ID", "CouchRevision", "CouchbaseCAS" ]

task 物件

"task": [ "JobID", "TaskID", "FrameString", "SlaveName", "Status", "Progress", "RenderStatus", "ErrorCount", "TaskStartTime", "RenderStartTime", "CompletedDateTime", "Name", "IsStarting", "WaitingToStart", "RenderTime", "TaskTime", "RenderTimeMultiplier", "NormalizedRenderTime", "ImageFileSize", "PeakRamUsage", "PeakRamPercentage", "PeakCpuUsage", "CpuUtilisation", "AverageRam", "AverageRamPercentage", "AverageSwap", "PeakSwap", "UsedCpuClocks", "TotalCpuClocks", "TaskId", "TaskFrameString", "TaskFrameList", "TaskStatus", "TaskRenderStatus", "TaskAverageRam", "TaskAverageRamPercentage", "TaskAverageSwap", "TaskCompletedDateTime", "TaskCpuUtilisation", "TaskErrorCount", "TaskOutputFileSize", "TaskIsStarted", "TaskJobId", "TaskName", "TaskNormalizedRenderTime", "TaskPeakCpuUsage", "TaskPeakRamPercentage", "TaskPeakRamUsage", "TaskPeakSwap", "TaskProgress", "TaskRenderStartTime", "TaskRenderTime", "TaskRenderTimeMultiplier", "TaskSlaveName", "TaskTotalCpuClocks", "TaskUsedCpuClocks", "TaskWaitingToStart", "TaskProperties", "ID", "CouchRevision", "CouchbaseCAS" ]