The project consists of a frontend website made with React+Tailswift CSS and a backend being a PostgreSQL database via SQLalchemy ORM. This database stores some global information that cannot be modified by users of the website like information about courses offered at IITK, the standard templates and credit requirement information; and then it also stores user specific data, where the user has full reign and can modify however he wishes. The global data needs to be filled in the database via an admin through a loading process consisting of automated scraping and storage from various data sources, followed by manual modifications by the admin wherever the automation wasn’t good enough.
The Main Data Sources are:
The website is scraped using selenium, and the pdf documents using pdfplumber. After appropriate parsing and data filtering, relevant data is committed to the database, having the following schema:
The bird’s eye view is that a student’s academic programme (called template here) constitutes of semesters, and each semester constitutes courses. The base templates are the suggestions offered by IITK for the timely completion of the course, however we have the liberty to add, switch, remove courses from wherever in this base templates. Therefore, when a user logs in and creates his user template, I first fill out their history of courses already done by referring to the gradesheet, then fill the remaining courses by referring to the correct base template; and I store this template separately in the user_templates, user_semesters, user_courses and user_schedules tables. Now the user may modify whatever aspect of their template that they want and it will never affect the global base templates. Similarly, when a user decides to add a particular course, the system would refer to the courses table for information, and fill that in; but duplicate the info and store in user_courses (and user_schedules) so that it remains fully modifyable.