mad.sh / mad.config (Settings) ------------------------------ # Plugin System: This allows writing a script file that will be loaded / overlaid upon launch, allowing users to # override functions already in mad.sh with additional functionality # (ie. unzip after successful download, skip urls that are already in completed.txt) ## Additional Hook Functions added for easy of overriding: # * OnLoad(): Occurs after load mad.config / load plugins (prior to processing). # * BeginProcessing(): Occurs immediately after beginning processin of urls.txt (loops with Catnaps). # * PreProcessUrl(): occurs immediately after reading in an unprocessed url (^http) line to process. # * PostSuccessfulDownload(): occurs after a download success (is marked #OK# in the urls.txt). # * PostFailedDownload(): occurs after a download fails (is marked #FAIL# in the urls.txt). # * PostFailRetryDownload(): occurs after a download fails with a retry (is marked #RETRY# in the urls.txt). # * DoneProcessingAllUrls(): occurs after all the urls have finished processing (no flocks or other terms downloading). # * PostSuccessfulUpload(): occurs after an upload success (after upload completed ticket is created in ./completed/). # * PostFailedUpload(): occurs after an upload fails definitively -- #FAIL# in the uploads_processed.txt # * PostFailRetryUpload(): occurs after an upload fails with a retry (network drop, unexpected result) # * DoneProcessingAllUploads: occurs after alll the files have finished processing # @Default= (load no plugins) LoadPlugins="" * This takes a comma separated list of filenames in the plugins folder if you wish to load / use them (case-sensitive). ie. LoadPlugins="SkipOkUrlsInResultsTxt.sh,CatnapUntilCtrlC.sh" Folder Location: ---------------- /plugins folder should reside in the ScriptDir (ame directory as mad.sh script) Plugin Info: ------------- Plugins are just scripts that contain functions that you wish to use to add functionality or to override (code to do other stuff or more stuff). I have added 10 "hook" functions in optimal places that allows performing additional actions during the processing of urls in mad.sh. The easist way to start making a plugin to do something you want is to copy and past the current function in a script, and then code in your additions. These will not be overwritten by a mad.sh script update. Notes: ------ * Overriding one of the 5 main hooks will always be safe. - v1 Hooks (using the mad.sh function name) works, as I intend to keep functionality updates elsewhere, though this is no longer the recommended way as it will not allow other plugins to use the hook. ie. OnLoad() {} - v2 Hooks do not care as they will all be loaded and ran in the hook. ie. OnLoad_() {} * If you override a function in mad.sh, you may need to re-implement its core functionality in your override (v1 hook). * Prefer using local variables ie. (local datestamp=) instead of variables, and/or name them uniquely as plugin functions have access to all mad.sh vars and functions, and can overwrite variables (no bueno). I recommend prepending your pluginname, filename, or an abreviation of (ie. local ppunzipUrlName=${remote_url}, or anything of the likes). Unless your intent is to overwrite a mad.sh variable (ie. WorkDir -- then you could hook OnLoad_() {} and set it (WorkDir="/some/path/") !!----------- Important -----------!! * If you use a v1 Hook (override a function name in mad.sh), no other plugins will have access to that hook. ! Feel free to share any helpful plugins you make with the group. I will review anything on the thread to verify it, and possibly add it to the official bundle with plugins if approved. === Some ideas to get you started === --------------------------------------- * Unzip files after successfully downloaded (PostSuccessfulDownload or PostProcessUrl or DoneProcessingAllUrls if you wish to perform unzipping at the end of processing everything. * Auto ./mad.sh reset any #RETRY# failed downloads after the entire urls.txt is processed and run again. * Shutdown computer after completion of downloads (or beep, or whatever) -- DoneProcessingAllUrls. * Create your own log of completions or urls and status to a DB * Start another process on load of script OnLoad() hook (will likely want to check process isn't already running, as multi terminal mad.sh script will each run the OnLoad() hook -- $isRunning=$(ps -ef|grep 'procname') - This could auto start ./mad.sh clipmon to monitor for copied urls. * Keep track of every url you have already downloaded and skip processing again * Add special url fixing to bad urls copied / pasted -- ie. http://1fichier.com/?1234567890abcdefg?download * Rename / move files out to collection based off of keywords or foldernames * Sleep indefinitely after processing urls.txt -- waiting for urls.txt to be updated with more urls * Warn if drive space < specific amount * Process unzipped downloads and rename folder with (#p, #v) * Process unzipped downloads and auto create thumbs * ... you get the just of it