Software guidelines

Winning solutions need to be made available under a popular OSI-approved license in order to be eligible to prize money as well as to the  "HEP meets ML" award; a fact sheet has to be filled in as well. New deadline is 10th October .  Code submission is optional for other participants, but we encourage them to fill the fact sheet nevertheless by 3rd November. The forum should be used for any inquiries about this.

  1. Submission instructions : how to package your software is detailed on the hml-validate page.  (1) When you make your final entry, upload the software zip file with your result file at the bottom of the submission page on the Kaggle website using "Upload your model" in the Home menu (or make the zip file on an URL). The URL should then be indicated in the fact sheet.  (2) You can also post your solutions to the Kaggle competition forum for the other participants to read. Winners must post or link to their solutions within fourteen (14) days of being notified of their winning status.
  2. License: By submitting their software, the authors grant to the organizers a license to use it for the purpose of verifying the challenge results and carrying out post-challenge analyses on other data. The authors retain all rights to their software. The organizers will keep the software confidential. However, to be eligible for prizes, the winners will have to make their code available under a popular OSI-approved license.
  3. Libraries supported: The authors are responsible to include ALL the necessary libraries with their software. The software should be completely self-contained, unless agreed in advance with the organizers. The authors are responsible to get all necessary licenses and make sure no thrird party rights are violated.
  4. Platforms supported:
    • -- Executable code -- The authors can provide an executable for either Windows 7, Mac OS X 10.5 or later, Linux slc5 or higher. We also recommend that the authors supply source code in addition to their executable, to be used only if the organizers fail to run the executable.
    • -- Interpretable code -- The authors can provide interpretable code for Matlab (release R2011a or higher), Mathematica (release 8.0.4.0, without additional toolboxes), Java (latest release), Python (release 2.7.2 and higher), or R (latest release) or Root (release 5.32 and higher)
    • -- Other platforms -- The authors should contact the organizers in advance if they want to submit compilable source code under Unix/Linux platforms or other versions of Windows or Mac OS, use other version of Matlab, Java, or Python, use other interpreted languages, use Matlab p-code, use particular Matlab toolboxes or Java and Python libraries.
  5. Recommendations:
    • -- Self-contained package -- Include all necessary libraries.
    • -- Syntax -- See details in the hml-validate page.
    • -- Absolute reproducibility -- Make sure that the code always returns the same result and does not have any stochastic component. If your code relies on random numbers, draw in advance a sequence of numbers and always use the same one in the same order.
    • -- Exception handling -- The code should possess some kind of exception handling such that it can't fail on any bona fide training or test sample.
    • -- File processing status -- The program should regularly output status information about progress of data processing.
    • -- Termination -- The program should end with a message stating that it exited normally indicating the full path and name of the file where the results were written to.
    • -- Documentation -- Add a README file with: a contact name and email, your code version and date, installation instructions, type and versions of OS, version of platform (e.g. Matlab) and libraries, usage instructions, compute power you used (processors, memory, disk space) and how long it took to process the validation data, the performance your obtained with that version of the code on validation data (with a lot of digits).
    • -- Data -- Do NOT include the data with your software. Contact the organizers if you used extra training data not provided by the organizers and want to donate them.
    • -- Test script -- The test script should be able to run the evaluation and (if required) the training. See details in the hml-validate page.
  6. Output format: Output file should be the submission file identical to the one you submitted. See details in the hml-validate page.

Recent Posts