Repository: mitodl/mitxpro
Pull request #3431: docs: add external course sync documentation
Author: login: asadali145
Labels: name: Waiting on Author
Created: 2025-03-12T09:13:01Z | Merged: 2025-03-26T08:23:54Z

### What are the relevant tickets?
<!--- If it fixes an open issue, please link to the issue here. -->
<!--- Closes # --->
<!--- Fixes # --->
<!--- N/A --->
None

### Description (What does it do?)
<!--- Describe your changes in detail -->
Adds external course sync documentation to explain what we do during the external course sync.





Diff:
diff --git a/courses/docs/external-course-sync.md b/courses/docs/external-course-sync.md
new file mode 100644
index 0000000000..ed82c99623
--- /dev/null
+++ b/courses/docs/external-course-sync.md
@@ -0,0 +1,142 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses with our system.
+
+### Setup Guide
+
+The sync will not work if the platform doesn't exist in Django admin or enable sync checkbox is unchecked.
+
+### External Course APIs
+
+Below is a list of external vendor APIs which sync with our system:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data.
+If any required field is missing, that external course is skipped.
+These fields are required for creating a course, course run, product, and product version.
+
+**Required Fields:**
+
+1. `course_title`
+2. `course_code`
+3. `course_run_code`
+4. `list_currency`
+
+### Currency Validation
+
+Ensures the listing currency is `USD` (because the system only supports USD). If not, the external course is skipped.
+
+### Course Run Dates Validation
+
+Ensures that the course run dates are valid. If not, the external course is skipped.
+
+- `start_date` must be provided and in the future.
+- `end_date` must be later than `start_date` and in the future.
+
+### Course Creation
+
+Creates the course based on the external course data.
+
+**Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+#### Creation
+
+- If the course run does not exist, it is created.
+  - If `start_date` and `end_date` are present in the external data but `enrollment_end` is not present,
+    then `enrollment_end` date is set to the `start_date`. This is done because external courses do not
+    allow enrollments after the start date, and setting the enrollment end date to the start date will hide
+    the course run from the course details page.
+
+#### Update
+
+- If the course run exists, it is updated with new data if necessary.
+  - Fields Updated:
+    - `start_date`
+    - `end_date`
+    - `enrollment_end`
+    - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run
+**if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update (CMS)
+
+#### Creation
+
+- If the external course page does not exist, it is created.
+
+#### Update
+
+- If the external course page exists, it is updated with new data if necessary.
+  - **Fields updated**:
+    - `external_marketing_url`
+    - `duration`
+    - `min_weeks`
+    - `max_weeks`
+    - `description`
+    - `background_image`
+    - `thumbnail_image`
+    - `language`
+- Background and thumbnail images are fetched from the existing images based on the image title
+  that we get in the external course data. The image titles are expected to match the `title` field of the Wagtail Image model.
+
+### Course Topic association
+
+Associates the course with the existing topics based on the course category that we get in the external course data.
+
+**Note:** We do not create new topics based on the external course data.
+
+### Learning Outcomes Page Creation
+
+If the learning outcomes page does not exist and there are learning outcomes in external course data, it is created.
+
+**Note:** We do not update the learning outcomes page after that.
+
+### Who Should Enroll Page Creation
+
+If who should enroll page does not exist and there are who should enroll items, it is created.
+
+**Note:** We do not update who should enroll page after that.
+
+### Certificate Page Creation or Update
+
+Creates or updates the certificate page for the course if CEUs are available in the external course data.
+
+#### Creation
+
+- If the certificate page does not exist, it is created.
+
+#### Update
+
+- If the certificate page exists, it is updated with new data if necessary.
+  - Updated Fields:
+    - CEUs
+
+### Course Overview Page Creation
+
+If the course overview page does not exist and there is a description in the external course data, it is created.
+
+### Common How You Will Learn and Enterprise Page Creation
+
+How You Will Learn and Enterprise pages are created if they do not exist.
+
+- These are created based on the common child pages feature where we have common pages for all courses.
+
+### Deactivation of Missing Course Runs and Products
+
+If a course run is not present in the external course data, it is deactivated along with the product.
diff --git a/mitxpro/utils.py b/mitxpro/utils.py
index 27091b1575..c8011f2120 100644
--- a/mitxpro/utils.py
+++ b/mitxpro/utils.py
@@ -675,11 +675,11 @@ def validate_courserun_dates(
     - expiration_date must be later than start_date and end_date (if provided).
 
     Args:
-        start_date (datetime or None): The start date of the course run.
-        end_date (datetime or None): The end date of the course run.
-        enrollment_start (datetime or None): The enrollment start date.
-        enrollment_end (datetime or None): The enrollment end date.
-        expiration_date (datetime or None): The expiration date of the course run.
+        start_date (datetime.datetime or None): The start date of the course run.
+        end_date (datetime.datetime or None): The end date of the course run.
+        enrollment_start (datetime.datetime or None): The enrollment start date.
+        enrollment_end (datetime.datetime or None): The enrollment end date.
+        expiration_date (datetime.datetime or None): The expiration date of the course run.
 
     Returns:
         tuple:


Reviews:
user: login: gemini-code-assist[bot]
body: Hello @asadali145, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

## Summary of Changes
This pull request adds documentation for the external course synchronization process. It outlines the steps involved in synchronizing external course data to create, update, or skip internal course models. The documentation covers aspects such as required field validation, currency validation, course/course run/product/external course page creation and updates, course topic association, and deactivation of missing course runs and products. It also provides references to the Emeritus and Global Alumni APIs.

### Highlights
* **External Course Sync Process**: The documentation details the steps involved in synchronizing external course data, including validation and creation/update processes.
* **API References**: The documentation provides links to the Emeritus and Global Alumni external course APIs.
* **Validation Steps**: The documentation outlines the required field and currency validation steps performed during the synchronization process.
* **Creation and Update Logic**: The documentation describes the logic for creating and updating courses, course runs, products, and external course pages based on external course data.

### Changelog
* **courses/docs/external-course-sync.md**
  * Added documentation outlining the external course synchronization process.
  * Documented the required field and currency validation steps.
  * Described the logic for creating and updating courses, course runs, products, and external course pages.
  * Included information on course topic association and learning outcome/certificate page creation.
  * Added details on deactivation of missing course runs and products.


<details>

<summary><b>Using Gemini Code Assist</b></summary>
<br>

The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips.

<b>Invoking Gemini</b>

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands.

Feature | Command | Description
--- | --- | ---
Code Review | `/gemini review` | Performs a code review for the current pull request in its current state.
Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state.
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments.
Help | `/gemini help` | Displays a list of available commands.

<b>Customization</b>

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/review-github-code).

<b>Limitations & Feedback</b>

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments to provide feedback.

</details>

<b>You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist).</b>


[^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/review-github-code). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).

---

**Did you know?**

The first version of Markdown was created in 2004 by John Gruber.

state: COMMENTED
author_association: NONE
submitted_at: 2025-03-12T09:13:09Z
user: login: gemini-code-assist[bot]
body: ## Code Review
This pull request adds documentation for the external course sync process. The documentation is well-structured and provides a good overview of the synchronization steps. However, there are a few areas where more detail or clarification could be beneficial.

### Merge Readiness
The documentation provides a useful overview of the external course sync process. While the identified areas for improvement are not critical, addressing them would enhance the documentation's clarity and completeness. I am unable to approve this pull request, and other reviewers should review this code before merging. I recommend addressing the medium severity issues before merging.
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-12T09:13:22Z
user: login: cachob
state: APPROVED
author_association: NONE
submitted_at: 2025-03-13T15:28:49Z
user: login: arslanashraf7
body: Some quick changes.

In addition, I was thinking that we should link to some sample pages when we say certificate page or overview page etc we can link an actual cms page for reference.
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T10:15:54Z
user: login: asadali145
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T10:42:12Z
user: login: asadali145
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T11:40:40Z
user: login: asadali145
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T11:40:55Z
user: login: arslanashraf7
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T12:19:15Z
user: login: arslanashraf7
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T08:13:13Z

Review comments:
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
path: courses/docs/external-course-sync.md
user: login: gemini-code-assist[bot]
body: ![medium](https://img.shields.io/badge/priority-medium-yellow)

Consider adding a brief explanation of what these fields represent or why they are required. This would provide more context for readers unfamiliar with the external course data structure.

```suggestion
Required Fields: `course_title` (Name of the course), `course_code` (Unique identifier for the course), `course_run_code` (Unique identifier for the course run), `list_currency` (Currency in which the course is listed)
```
created_at: 2025-03-12T09:13:21Z
author_association: NONE
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. The reason behind that is that course runs with past start dates are not displayed on the external course site.
path: courses/docs/external-course-sync.md
user: login: gemini-code-assist[bot]
body: ![medium](https://img.shields.io/badge/priority-medium-yellow)

It would be helpful to clarify why the enrollment end date is set to the start date. Is this a business rule, or a technical limitation? Adding a brief explanation would improve understanding.

```suggestion
  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
```
created_at: 2025-03-12T09:13:22Z
author_association: NONE
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. The reason behind that is that course runs with past start dates are not displayed on the external course site.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run **if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update
+
+- If the external course page does not exist, it is created.
+- If the external course page exists, it is updated with new data if necessary.
+- Updated Fields:
+  - external_marketing_url
+  - duration
+  - min_weeks
+  - max_weeks
+  - description
+  - background_image
+  - thumbnail_image
+  - language
+- Background and thumbnail images are fetched from the existing images based on the image title that we get in the external course data.
path: courses/docs/external-course-sync.md
user: login: gemini-code-assist[bot]
body: ![medium](https://img.shields.io/badge/priority-medium-yellow)

Consider adding a sentence or two explaining how the image titles are used to fetch the images. Are there specific naming conventions or expected formats?

```suggestion
- Background and thumbnail images are fetched from the existing images based on the image title that we get in the external course data. The image titles are expected to match the `title` field of the Wagtail Image model.
```
created_at: 2025-03-12T09:13:22Z
author_association: NONE
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
This document outlines the checks performed during the synchronization of external courses with our system.
```
created_at: 2025-03-21T09:46:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
Below is a list of external vendor APIs which sync with our system:
```
created_at: 2025-03-21T09:48:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
Ensures that all required fields are present in the external course data. If any required field is missing, that external course is skipped.
```
created_at: 2025-03-21T09:51:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
**Required Fields:**
1. course_title
2. course_code
3. course_run_code
4. list_currency

```
created_at: 2025-03-21T09:53:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
Ensures the listing currency is `USD` (because the system only supports USD). If not, the external course is skipped.
```
created_at: 2025-03-21T09:55:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: We might want to tell a little about why title and course codes are required. This will help the reader understand the required fields.
created_at: 2025-03-21T09:56:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: This document should also include the basic setup guide. e.g. The sync will not work if the platform doesn't exist in Django admin or `enable sync` checkbox is unchecked.
created_at: 2025-03-21T09:59:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
Ensures that the course `end_date` is in the future. If not, the external course is skipped.
```
created_at: 2025-03-21T10:01:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: Similar suggestions can be applied in other places, to reduce the length of these lines and keep this readme concise. 
created_at: 2025-03-21T10:01:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: I think we do, we mark the courses inactive based on their dates I think?
created_at: 2025-03-21T10:02:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
  - If `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then `enrollment_end` date is set to the `start_date`. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date will hide the course run from the course details page.
```
created_at: 2025-03-21T10:06:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: We should have `Creation` and `Updation` headings here. And related sections will move underneath.
created_at: 2025-03-21T10:07:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
- If the course run exists, it is updated with new data if necessary.
    - Updated Fields:
      - `start_date`
      - `end_date`
      - `enrollment_end`
      - `live`
```
created_at: 2025-03-21T10:09:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run **if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: ```suggestion
### External Course Page Creation or Update (CMS)
```
created_at: 2025-03-21T10:11:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run **if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update
+
+- If the external course page does not exist, it is created.
+- If the external course page exists, it is updated with new data if necessary.
+- Updated Fields:
+  - external_marketing_url
+  - duration
+  - min_weeks
+  - max_weeks
+  - description
+  - background_image
+  - thumbnail_image
+  - language
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: This and Update fields(commented above) should be highlighted more, maybe they should have their own heading or something ng but these are the most important details here.
created_at: 2025-03-21T10:12:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run **if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update
+
+- If the external course page does not exist, it is created.
+- If the external course page exists, it is updated with new data if necessary.
+- Updated Fields:
+  - external_marketing_url
+  - duration
+  - min_weeks
+  - max_weeks
+  - description
+  - background_image
+  - thumbnail_image
+  - language
+- Background and thumbnail images are fetched from the existing images based on the image title that we get in the external course data. The image titles are expected to match the `title` field of the Wagtail Image model.
+
+### Course Topic association
+
+Associates the course with the existing topics based on the course category that we get in the external course data.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: Also need to highlight explicitely that we don't auto create new topics based on external data.
created_at: 2025-03-21T10:13:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run **if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update
+
+- If the external course page does not exist, it is created.
+- If the external course page exists, it is updated with new data if necessary.
+- Updated Fields:
+  - external_marketing_url
+  - duration
+  - min_weeks
+  - max_weeks
+  - description
+  - background_image
+  - thumbnail_image
+  - language
+- Background and thumbnail images are fetched from the existing images based on the image title that we get in the external course data. The image titles are expected to match the `title` field of the Wagtail Image model.
+
+### Course Topic association
+
+Associates the course with the existing topics based on the course category that we get in the external course data.
+
+### Learning Outcomes Page Creation
+
+If the learning outcomes page does not exist and there are learning outcomes in external course data, it is created.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: Is it ever updated too? If not, we should highlight that.
created_at: 2025-03-21T10:14:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
+
+- If the course does not exist, it is created. Otherwise, we get it from the database.
+
+### Course Run Creation or Update
+
+Creates or Updates the course run based on the external course data.
+
+- If the course run does not exist, it is created.
+  - `start_date` and `end_date` are present in the external data but `enrollment_end` is not present, then enrollment end date is set to the start date. This is done because external courses do not allow enrollments after the start date, and setting the enrollment end date to the start date hides the course run from the course details page.
+- If the course run exists, it is updated with new data if necessary.
+- Updated Fields:
+  - `start_date`
+  - `end_date`
+  - `enrollment_end`
+  - `live`
+
+### Product and Product Version Creation or Update
+
+Creates or updates the product and product version for the course run **if price is available in the external course data**
+
+- If the product does not exist, it is created. We mark it as active when we update it.
+- If the price does not exist or is different from the external course price, a new product version is created.
+
+### External Course Page Creation or Update
+
+- If the external course page does not exist, it is created.
+- If the external course page exists, it is updated with new data if necessary.
+- Updated Fields:
+  - external_marketing_url
+  - duration
+  - min_weeks
+  - max_weeks
+  - description
+  - background_image
+  - thumbnail_image
+  - language
+- Background and thumbnail images are fetched from the existing images based on the image title that we get in the external course data. The image titles are expected to match the `title` field of the Wagtail Image model.
+
+### Course Topic association
+
+Associates the course with the existing topics based on the course category that we get in the external course data.
+
+### Learning Outcomes Page Creation
+
+If the learning outcomes page does not exist and there are learning outcomes in external course data, it is created.
+
+### Who Should Enroll Page Creation
+
+If the who should enroll page does not exist and there are who should enroll items, it is created.
path: courses/docs/external-course-sync.md
user: login: arslanashraf7
body: Same here.

created_at: 2025-03-21T10:14:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
+
+Ensures that the price is in `USD`. Only USD is supported. If the currency is not USD, the external course is skipped.
+
+### End Date Validation
+
+Ensures that the course `end_date` is in the future. If the course end date is in the past, the external course is skipped.
+
+### Course Creation
+
+Creates the course based on the external course data. **Note:** We do not update the course object after that.
path: courses/docs/external-course-sync.md
user: login: asadali145
body: We deactivate course runs, not courses.
created_at: 2025-03-25T10:42:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
path: courses/docs/external-course-sync.md
user: login: asadali145
body: Added
created_at: 2025-03-25T11:40:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,100 @@
+## External Course Sync Documentation
+
+This document outlines the checks performed during the synchronization of external courses to create, update, or skip the internal models.
+
+### External Course APIs
+
+For reference, the external course APIs are:
+
+- [Emeritus API](https://xpro.mit.edu/api/external_courses/emeritus/)
+- [Global Alumni API](https://xpro.mit.edu/api/external_courses/global_alumni/)
+
+### Required Fields Validation
+
+Ensures that all required fields are present in the external course data. If any of the required fields are missing, the external course is skipped.
+
+Required Fields: `course_title, course_code, course_run_code, list_currency`
+
+### Currency Validation
path: courses/docs/external-course-sync.md
user: login: asadali145
body: Added.
created_at: 2025-03-25T11:40:55Z
author_association: CONTRIBUTOR

PR comments:
user: login: asadali145
created_at: 2025-03-12T09:33:22Z
body: End date validation docs will be updated once we merge https://github.com/mitodl/mitxpro/pull/3400
author_association: CONTRIBUTOR
user: login: asadali145
created_at: 2025-03-25T11:43:23Z
body: > In addition, I was thinking that we should link to some sample pages when we say certificate page or overview page etc we can link an actual cms page for reference.

I don't think it should be a part of the readme. pages can be deleted or updated and some people might not have access as well.
author_association: CONTRIBUTOR
user: login: asadali145
created_at: 2025-03-25T16:09:41Z
body: @arslanashraf7, could you please have a look at the last commit: https://github.com/mitodl/mitxpro/pull/3431/commits/372fe5f65cdcfdbef40989bd60949d1e9bbe43f4? I have rebased and updated some of the docs as extra validations are applied on course run dates after merging https://github.com/mitodl/mitxpro/pull/3400.
author_association: CONTRIBUTOR