Virtual Labs Hosting Process

Introduction

Virtual Labs is a mission mode project initiated by the Ministry of Human Resources and Development (MHRD). The objective of this project is to provide laboratory learning experience to the students who do not have access to adequate laboratory infrastructure. Currently there are around 90 labs developed by various institutes in the consortium. A streamlined software development life cycle process followed for the development of these labs ensures high quality labs. This document defines the hosting process to be followed by the developers (open source community) of the Virtual Labs project.

Purpose

This document defines the experiment on-boarding and hosting process that should be followed by the developers of the lab while requesting the hosting of their lab on AWS. The roles and responsibilities of the various parties involved in the hosting process is also discussed in detailed in the document.

Motivation

A well defined experiment hosting and on-boarding process will help maintain a consistent user experience and enable experiment-authors to focus on the content. Consolidated information regarding all the deployments will also facilitate reporting.

Audience

The target audience for this document is the hosting team at CPE, IIITH and all the lab authors and owners who want to avail the hosting service provided by Central Platform Engineering Team (CPE), IIITH team.

Definitions and Pre-requisites

Virtual Labs hosting process will require certain prerequisites to be met by the experiments that need to be hosted. The following paragraphs define terminologies used during the hosting process.

Hosting

Hosting is a service provided by the Central Platform Engineering (CPE) which includes the following:

  1. Providing infrastructure including all dependencies required to run the Lab/Experiments as specified in the Onboarding request of the hosting unit.
  2. Ensuring round the clock availability of the Lab/Experiments.
  3. Ensuring that a minimum acceptable performance criteria are met by the Lab/Experiments.

    Following are not part of the hosting service:

  4. Testing: The deployable code is a black box for the CPE team. The CPE team is not responsible for testing the deployments. The developer/owner of the Lab/Experiment is responsible for confirming that the Lab/Experiments are deployed and functioning as intended. This confirmation is part of the hosting process.
  5. Implementation: The CPE team is not responsible for any bug fixes or enhancements or implementation on the Lab/Experiment. The developers should ensure that their product is complete and functional before raising a hosting request.
  6. Approvals: The CPE team does not provide any approvals regarding the completeness of the Lab/Experiment. The developers are expected to have sought the necessary approvals regarding completeness of the Lab/Experiment before raising a hosting request.

Hosting Unit

Hosting unit is the code base/repository that will hosted in its enirety during the hosting process. It will not be possible for a part of a hosting unit to be hosted separately. A hosting unit will correspond to one public repository.A hosting unit could be a stand alone experiment or could be a lab consisting of a set of related experiments.

Repository Owner

Each repository, corresponding to a hosting unit, will have one designated owner. The owner will be the single entity responsible for all the code that resides in the repository. The owner will also be the final approval authority for a hosting request. The owner should ensure that the master branch of the repository only contains the code that needs to be hosted. The latest code in the master branch should represent the hosted experiment.

The repository owner does not have to be a person. It may be any uniquely identifiable entity. From the perpective of Virtual Labs hosting team, this entity will the single point of contact for all clarifications. The hosting team will not be responsible for any confusions arising due to shared accessibility of the repo owner entity.

Requester

A repository owner or a developer who raises a hosting request.

Tags

The repository owner will be responsible for tagging each merge to the master branch. The owner will have to follow Semantic Versioning . In short, each version is a combination of three numbers (MAJOR.MINOR.PATCH) separated by dots. The changes to these numbers represent the following:

  1. MAJOR: A change incompatible with previous versions.
  2. MINOR: A backwords compatible new feature.
  3. PATCH: A backwords compatible bug fix.

All Virtual Labs hosting request should specify a tag.

Build Process

Each hosting unit should have a well defined build process. The hosting team will treat the build process as a black box action. This action will be triggerred to generate the artifacts to deploy. The repository owner should ensure that the build process is well documented and functioning as desired before raising a hosting request.

Testing

The hosting process will result in a deployment with an accessible url. The responsibility of testing the deployments will lie with the repository owner. The hosting team will be responsible for providing the link to the deployment. The hosting team will not provide any testing environment. The repository owner will be responsible for ensuring that the hosting unit works as expected in their own specific testing environments.

Hosting Process

Virtual Labs hosting process consists of two steps -

  1. Request the onboarding of a hosting unit
  2. Request the hosting of hosting unit

On-Boarding process of hosting unit

Each hosting unit that needs to be hosted by the Virtual Labs hosting team, must follow an onboarding process. The repository owner of the hosting unit will need to raise an issue of type 'on-boarding request' in the engineers-forum. The issue will need to filled with the following pieces of information :

On-Boarding Request -

  1. A non-conflicting name for the hosting unit.
  2. Repository URL for the hosting unit.
  3. A designated repository owner with the following details:

    a. Repository owner github id

    b. Repository owner email address

  4. The build command for the hosting unit along with the link to the build process documentation.
  5. Hardware specification including:

    a. CPU

    b. Memory

    c. Network bandwidth

  6. Software environment specification including:

    a. Host operating system with minimum compatible version

    b. Any additional software packages with minimum compatile versions

    c. Browser plugins required to view experiment

    d. DB

Repository owner will be reponsible for keeping the above information current and intimating the hosting team of any updates/changes as and when needed.

On receipt of an onboarding request the hosting team will verify the data and acknowledge the same as part of the request.

The hosting team will be responsible for storing the this information at a central location and making it available for all stakeholders.

Hosting Process of hosting unit

A repository owner/developer (requester ) will have to raise an issue of type 'hosting request' in the GitHub repository engineers-forum under Virtual-Labs organisation for hosting of a hosting unit. The issue will need to filled with the following pieces of information :

  1. Repository URL
  2. Branch to be deployed
  3. Tag to be deployed
  4. Contact email address and github handle to be used in cases of clarifications

A hosting request will be tied to this information. This issue will be a single source of truth for that hosting request. All communication related to the hosting will be recorded on the raised hosting request.

Lifecycle of a Hosting Request

The hosting request will go through the following lifecycle:

  1. Requester will raise a hosting request issue by duly filling in the details in the 'hosting request'.
  2. The hosting team will attach the url of the corresponding onboarding request to the hosting request. If the onboarding request is not found ( by searching on the repo url), then the hosting team will add its findings and label the issue as Failed.
  3. If requester is not the repository owner, the hosting team will seek approval from the repository owner. This will be through the above created GitHub hosting request issue.
  4. The hosting team will honour the hosting request only on receiving the approval from repository owner else will label the hosting request issue as Not Approved
  5. On a successful hosting, the hosting team will add the link of the newly hosted content to the hosting request issue. They will also label the issue as Hosted.
  6. However, on failure ( because of missing or incorrect information or repository problems,), the hosting team will label the issue as Failed.
  7. The requester will be responsible for fixing the failed hosting bug and will need to raise a new 'hosting request' to get the unit hosted.
  8. Once the hosting is successful, the requester will be responsible for the verification of the hosted link. If the hosting is not as expected, the requester should change the label from Hosted to Reopened. The requester will also need to specify if he/she would like to revert to the previous hosted image.
  9. An issue with Reopened label will by default host the repository and branch/tag as specified in the hosting request initially. However, if the requester has specifically requested a revert, the previously hosted image will be restored and issue will be labelled as Reverted.
  10. If the requester wants to revert to any earlier branch/tag, a new 'hosting request' will need to be raised to get the unit hosted.
  11. Requester will be responsible for closing a hosting request issue by changing the issue status from Open to Closed when the issue is labelled as Hosted or Reopened or Reverted or Not Approved.

Labels & Status of a Hosting Request

At any given time a hosting issue should be marked with only one of the following labels. To change the label of an issue, the current label of the issue should be unchecked and the new label should be checked.

Hosted : This label indicates that the hosting request has been successful and the hosted url has been shared in the hosting request issue. This label is used only by the hosting team.

Failed : This label indicates that the hosting request has not been successful. This label is used only by the hosting team.

Reopened : This label is used by the requester to indicate that the hosting url provided on a successful hosting of the hosting unit is not passing the validation.

Not Approved : This label indicates that the hosting request issue raised by the requester was not approved by the Repository Owner. This label is used only by the hosting team.

Reverted : This label indicates that the hosting request has been successfully revert to the previous hosted image. This label is used only by the hosting team.

Apart from the above mentioned labels a hosting request issue can either be in a closed or open status as provided by GitHub.

Open : This status indicates that a new hosting request issue has been raised and is waiting for the services of the hosting team. All issues are in this status when they are created on GitHub.

Closed : This status indicates that a hosting request has been serviced by the hosting team and is labelled as Hosted or Reopened or Reverted or Not Approved.

Central Hosting Data

The following information will be stored at a central place for a quick reference by the hosting team:

  1. Hosting unit name
  2. Repository URL
  3. Currently hosted Branch/Tag
  4. Previously hosted Branch/Tag
  5. Date/Time of hosting

Please follow the link for the Central Hosting Data.

Flow Diagram of the Hosting Process

The flow diagram depicting lifecycle of a hosting request is below:

hosting-process.png

edit image

Roles and Responsibilities

FAQ

Why should I use the Hosting Service?

To get more for nothing. Hosting Service provides you a high-availability, high-performance deployment environment. You do not have to spend time, money, people or other resources to ensure that your website is available to your target audience. You can also take advantage of other services, like Analytics, provided by the Central Platform Engineering team and monitor multiple statistics about the users and usage patterns of your Lab.

Do I get better website performance if I use Hosting Service?

Almost all the time. The CPE team works continuously to improve the overall and individual Lab/Experiment performance based on the Analytics data. You can compare the performance of the Labs/Experiments deployed through the hosting service to those deployed elsewhere. If you use the Analytics service integration, the CPE team can individually target your Lab/Experiment for better performance.

Does the CPE team help me improve my Lab performance?

Yes. The CPE team continuously works to improve the infrastructure in order to achieve best performance results for your Lab. The CPE team does not improve your code, however, it can provide you guidelines and actionable performance improvement advice.

Does Hosting Service add a UI to my webpages?

No. Hosting service does not alter your code in any way. As the developer, you are responsible for ensuring that your Lab/Experiment is complete in all respects before you request it to be hosted. Hosting service is strictly limited to its stated objectives.

How can I get Analytics for my experiments?

By working with the CPE team. Analytics service is another service provided by the CPE team. You can work with the CPE team to integrate Analytics service into your Labs. You can also get access to customized dashboards/reports in order to monitor multiple statistics about the users and usage patterns of your Lab.

Does the CPE team test my Lab for correctness as part of hosting?

No. It is not possible for the CPE team to test your Lab because only you understand the right definition of correctness for your Lab/Experiment. As part of hosting, the CPE team ensures that your Lab is up and accessible. You are responsible for ensuring the completeness and correctness of your code and deployment.

Will the CPE team tell me about broken links as part of hosting?

No. Your code is a complete blackbox for the CPE team. You are responsible for ensuring the completeness and correctness of your code and deployment.

Conclusion

This document highlights the hosting process of Virtual Labs from on-boarding to hosting. It aims at collaborating contributions of the developers across the open source platform - GitHub. This streamlined process would help in our strive for excellence in delivering high standard labs.

Glossary

  • Repository : The distributed version control system - git is used here. Every time the term repository or repo is used, it refers to a git repository. A repository is an on-disk data structure which stores metadata for a set of files and/or directory structure. The whole set of information in the repository may be duplicated on every user's system or may be maintained on a single server.
  • Virtual Labs : Virtual Labs is an initiative by MHRD under NMEICT. The objective of this project is to make engineering education engaging, enjoyable, immersive and online.
  • Hosting team: VLEAD at IIITH which provides Virtual Labs hosting service

Pages generated from emacs org-mode