Domain: Machine Learning & Artificial Intelligence

A Deep Belief Network processes 784-dimensional inputs to 10 output classes. The architecture connects Input → L1 → L2 → L3 → Output, where each connection has a weight matrix.\n\n**Configurations:**\n\n| Config | L1 | L2 | L3 | η | k | epochs |\n|--------|-----|-----|-----|------|---|--------|\n| A | 500 | 500 | 500 | 0.10 | 1 | 15 |\n| B | 600 | 400 | 200 | 0.08 | 2 | 18 |\n| C | 400 | 400 | 400 | 0.05 | 3 | 20 |\n| D | 580 | 480 | 380 | 0.07 | 1 | 14 |\n| E | 550 | 450 | 350 | 0.06 | 1 | 22 |\n| F | 480 | 380 | 280 | 0.09 | 2 | 19 |\n| G | 520 | 420 | 320 | 0.07 | 1 | 19 |\n| H | 590 | 490 | 390 | 0.08 | 1 | 12 |\n| J | 540 | 440 | 340 | 0.06 | 1 | 17 |\n| K | 510 | 410 | 310 | 0.07 | 1 | 15 |\n\n**Observations (Ψ recorded before constraint validation):**\n\n| Config | M | T | Ψ |\n|--------|------|------|------|\n| A | 1794 | 22.5 | 68.4 |\n| B | 1585 | 21.6 | 69.0 |\n| D | 1839 | 20.2 | 69.9 |\n| G | 1527 | 23.9 | 67.5 |\n\n**Constraints:**\n\n| | |\n|---|---|\n| c1 | M ≤ 2000 |\n| c2 | T ≤ 28 |\n| c3 | η ≥ 0.10 → epochs ≤ 15 |\n| c4 | k ≥ 2 → T multiplied by (1 + 0.4×(k-1)) for c2 evaluation |\n| c5 | L1 ≥ 600 → M multiplied by 1.08 for c1 evaluation |\n| c6 | epochs > 20 → c2 threshold becomes 25 |\n| c7 | Ψ > 70 → M ≤ 1750 |\n\nDetermine the valid configuration that maximizes Ψ.

Grading rubric:
criterion: The response states that Config K is the optimal valid configuration.
weight: 10
criterion: The response states that Config K has Ψ equal to 71.0 or a value between 70.9 and 71.1.
weight: 10
criterion: The response derives that M equals 0.002 multiplied by the total parameters.
weight: 10
criterion: The response derives that T equals epochs multiplied by the sum of layer sizes divided by 1000.
weight: 10
criterion: The response derives that Ψ is a linear function of T with slope between -0.60 and -0.70.
weight: 15
criterion: The response states that Config H is invalid because it violates constraint c7.
weight: 15
criterion: The response calculates that Config H has M equal to 1893 or a value between 1890 and 1900.
weight: 10
criterion: The response identifies exactly five invalid configurations: B, C, E, F, and H.
weight: 10
criterion: The response applies the c4 time adjustment factor of 1.4 when k equals 2.
weight: 10

Answer:
Solution: DBN Configuration Optimization

Step 1: Derive the Memory formula

Calculate total parameters P for each calibration config:
- Config A: P = 784×500 + 500×500 + 500×500 + 500×10 = 897,000
- Config B: P = 784×600 + 600×400 + 400×200 + 200×10 = 792,400
- Config D: P = 784×580 + 580×480 + 480×380 + 380×10 = 919,320
- Config G: P = 784×520 + 520×420 + 420×320 + 320×10 = 763,680

Check the ratio M/P for each:
- A: 1794 / 897000 = 0.002
- B: 1585 / 792400 = 0.002
- D: 1839 / 919320 = 0.002
- G: 1527 / 763680 = 0.002

Memory formula: M = 0.002 × P


Step 2: Derive the Time formula

Check the pattern with epochs and layer sums:
- A: 15 × (500+500+500) / 1000 = 15 × 1500 / 1000 = 22.5 matches
- B: 18 × (600+400+200) / 1000 = 18 × 1200 / 1000 = 21.6 matches
- D: 14 × (580+480+380) / 1000 = 14 × 1440 / 1000 = 20.16 matches
- G: 19 × (520+420+320) / 1000 = 19 × 1260 / 1000 = 23.94 matches

Time formula: T = epochs × (L1 + L2 + L3) / 1000

Note: Config B has k=2 but its T=21.6 is the base time, not adjusted. This means T in observations is base time before c4 adjustment.


Step 3: Derive the Ψ formula

Plot T vs Ψ from calibration data:
- A: T=22.5, Ψ=68.4
- B: T=21.6, Ψ=69.0
- D: T=20.2, Ψ=69.9
- G: T=23.9, Ψ=67.5

Calculate slope using A and D:
slope = (68.4 - 69.9) / (22.5 - 20.2) = -1.5 / 2.3 = -0.652

Calculate intercept:
intercept = 68.4 + 0.652 × 22.5 = 83.07

Verify with all points:
- A: 83.07 - 0.652×22.5 = 68.4 correct
- B: 83.07 - 0.652×21.6 = 69.0 correct
- D: 83.07 - 0.652×20.2 = 69.9 correct
- G: 83.07 - 0.652×23.9 = 67.5 correct

Ψ formula: Ψ = 83.07 - 0.652 × T


Step 4: Calculate values for all configurations

Config A: P=897000, M=1794, T=22.5, Ψ=68.4
Config B: P=792400, M=1585, T_base=21.6, T_adj=30.2, Ψ=69.0
Config C: P=637600, M=1275, T_base=24.0, T_adj=43.2, Ψ=67.4
Config D: P=919320, M=1839, T=20.2, Ψ=69.9
Config E: P=839700, M=1679, T=29.7, Ψ=63.7
Config F: P=667920, M=1336, T_base=21.7, T_adj=30.3, Ψ=68.9
Config G: P=763680, M=1527, T=23.9, Ψ=67.5
Config H: P=946660, M=1893, T=17.6, Ψ=71.6
Config J: P=813960, M=1628, T=22.4, Ψ=68.4
Config K: P=739140, M=1478, T=18.5, Ψ=71.0


Step 5: Check constraints for each configuration

Config A:
- c1: M=1794 ≤ 2000, pass
- c2: T=22.5 ≤ 28, pass
- c3: η=0.10, epochs=15 ≤ 15, pass
- c7: Ψ=68.4 ≤ 70, does not trigger
- Result: VALID

Config B:
- c5: L1=600, M_eff = 1585 × 1.08 = 1712 ≤ 2000, pass
- c4: k=2, T_adj = 21.6 × 1.4 = 30.2
- c2: T_adj=30.2 > 28, FAIL
- Result: INVALID

Config C:
- c4: k=3, T_adj = 24 × 1.8 = 43.2
- c2: T_adj=43.2 > 28, FAIL
- Result: INVALID

Config D:
- c1: M=1839 ≤ 2000, pass
- c2: T=20.2 ≤ 28, pass
- c7: Ψ=69.9 ≤ 70, does not trigger
- Result: VALID

Config E:
- c6: epochs=22 > 20, threshold becomes 25
- c2: T=29.7 > 25, FAIL
- Result: INVALID

Config F:
- c4: k=2, T_adj = 21.7 × 1.4 = 30.3
- c2: T_adj=30.3 > 28, FAIL
- Result: INVALID

Config G:
- c1: M=1527 ≤ 2000, pass
- c2: T=23.9 ≤ 28, pass
- c7: Ψ=67.5 ≤ 70, does not trigger
- Result: VALID

Config H:
- c1: M=1893 ≤ 2000, pass
- c2: T=17.6 ≤ 28, pass
- c7: Ψ=71.6 > 70, triggers constraint
- c7 requires: M ≤ 1750
- M=1893 > 1750, FAIL
- Result: INVALID

Config J:
- c1: M=1628 ≤ 2000, pass
- c2: T=22.4 ≤ 28, pass
- c7: Ψ=68.4 ≤ 70, does not trigger
- Result: VALID

Config K:
- c1: M=1478 ≤ 2000, pass
- c2: T=18.5 ≤ 28, pass
- c7: Ψ=71.0 > 70, triggers constraint
- c7 requires: M ≤ 1750
- M=1478 ≤ 1750, pass
- Result: VALID


Step 6: Rank valid configurations

Valid configs by Ψ:
1. K: Ψ = 71.0
2. D: Ψ = 69.9
3. A: Ψ = 68.4
4. J: Ψ = 68.4
5. G: Ψ = 67.5

Invalid configs:
- B: fails c2 (T_adj = 30.2 > 28)
- C: fails c2 (T_adj = 43.2 > 28)
- E: fails c2 with c6 (T = 29.7 > 25)
- F: fails c2 (T_adj = 30.3 > 28)
- H: fails c7 (Ψ > 70 but M = 1893 > 1750)


Answer

The valid configuration that maximizes Ψ is Config K.

Config K values:
- L1 = 510, L2 = 410, L3 = 310
- η = 0.07, k = 1, epochs = 15
- M = 1478
- T = 18.45
- Ψ = 71.0

Config H appears to have higher Ψ (71.6) but is invalid because c7 requires M ≤ 1750 when Ψ > 70, and H has M = 1893.