Syllabus

Course Title: Optimization Models in Engineering

Units: 4

Course Format: 3 hours of lecture, 1 hour discussion per week

Contact: Please use Ed for all requests related to the course. Email the professor for any other requests ranade@eecs.berkeley.edu

Prerequisites: Math 54, CS 70, Math 53

Online Resources:

Website: This website includes all public course information, syllabus, resources, and schedule. Gradescope: for submitting HW and final exam scoring. Please join the course using the access code: 4DD74W . Note, it is your responsibility to make sure that the SID is exactly correct and that the entered email matches your official Berkeley email address in CalCentral. Otherwise, your grades might be lost and we may not be able to recover them. Ed Forum: We will use Ed as a forum for discussing course material and for sending out course announcements. If you haven’t been added, please join using this link: https://edstem.org/us/join/hAUaBC

Course Description

This course offers an introduction to optimization models and their applications, ranging from machine learning and statistics to decision-making and control, with emphasis on numerically tractable problems, such as linear or constrained least-squares optimization.

Desired Course Outcome

By the end of the term, students having taken Optimization Models in Engineering are expected to be able to:

1. Understand the basic concepts of linear algebra: vectors, matrices, rank, projections; symmetric matrices, positive semidefinite matrices, eigenvalues; singular value decomposition and principal component analysis.
2. Understand three basic optimization models: least-squares, linear programming, and quadratic programming; and develop an understanding of the more general convex optimization.
3. Be aware of the wide-ranging applications where optimization models are useful, including in machine learning applications.
4. Be able to use prototyping software such as CVX to develop optimization-based solutions in concrete applications.

Course Content

The course content will roughly follow the outline posted on the website, but is subject to change and evolve throughout the semester.

1. Eigenvalue and singular value decomposition
2. Linear equations
3. Least squares
4. Convexity
5. Gradient descent
6. Linear programming
7. Quadratic programming
8. Duality
9. Optimality conditions
10. Applications: machine learning
11. Applications: control

Some application topics might be added mid-semester. Relevant Textbooks include: Optimization Models, G.C. Calafiore and L. El Ghaoui, Cambridge University Press, October 2014. Boyd and Vandenberghe, https://stanford.edu/~boyd/cvxbook/ Lecture notes on Optimization, Pravin Varaiya, https://people.eecs.berkeley.edu/~varaiya/Download/Varaiya-Optimization.pdf Course Reader, https://eecs127.github.io/assets/notes/eecs127_reader.pdf

Discussion

Discussion sections will take place every week, and are intended to provide further practice with the material and an opportunity to ask questions in a smaller class environment. You may attend any section. Discussions will begin on January 21th (the first week of classes).

Wednesday 2-3 PM, Stanley 177 (Staff: Chris) Wednesday 3-4 PM, Dwinelle 219 (Staff: Syomantak) Wednesday 4-5 PM, Dwinelle 88 (Staff: Wayne) Wednesday 5-6 PM, Etcheverry 3107 (Staff: Wayne) Friday 2-3 PM, Etcheverry 3107 (Staff: Syomantak) Friday 4-5 PM, Lewis 9 (Staff: Chris)

Solutions to the discussion material will be released after the last discussion (on Friday).

Homework

We require students to not only do the homeworks, but to carefully understand the solutions. You are required to critically evaluate your homework and to make sure that you can actually do every problem on the assignment for your learning. To do this, we recommend that you redo any part of any question that you didn’t get right the first time in light of what you have learned from the solutions. There will be no self-grade submission.

Homework is worth 0% of the class grade. However, instructors may choose to look at the homework submissions in the event of a grade boundary case at the instructors discretion.

Homework will be assigned every Friday on the course website and will be due the following Friday night at 11:00pm. However, the Gradescope deadline will be the beginning of finals week. If you need a couple of extra days to complete the homework you may take them, but know that this comes at the risk of falling behind in the course. We will be teaching lectures and discussions at a pace that assumes that you have completed the weekly homeworks on schedule.

Homework exists mainly for you to learn. We encourage collaboration in teams of 2-4, but everyone must turn in their own submission. To submit please include:

1. Scanned handwritten work or a LaTeX file containing answers to non-Jupyter notebook questions.
2. A PDF of all Jupyter notebooks. The deadline for submitting homeworks will be set to the last day of RRR week for all assignments. Homework will not be graded.

The recommended process for doing the homework is: (0) Skim the problems; (1) Start doing the problems on your own, noting down where you get stuck but not letting yourself get stuck for more than 3-5 minutes on anything; (2) Go back to do parts that you are stuck on by yourself or with your study group — talking through where you are stuck and getting past it — this is also the phase in which you can ask questions on Ed as well as look to see what people have posted there; (3) If you talked to anyone, go back and write out your solutions on your own without access to your notes — in effect, make sure you can do it yourself. Refer to your notes to unstick yourself; (4) After you’ve finished doing the homework yourself, try to engage with GenAI tools like Gemini, ChatGPT, etc. and see if you can use them to increase the depth of your understanding of the problems and the topics; (5) Once the solutions are released by the staff, check your own work carefully. Note what you got right, what you missed or misunderstood, etc.; (6) Redo any problem that you didn’t get without looking at the solutions. If you get stuck, refer to the solution as a hint but then do the problem without looking at it. You need to make sure you can do the problems straight — because on the exam, you won’t have anything else; (7) If anything is still unclear from the solution and your attempts to redo the questionask on Ed in the solutions thread.

Exam Policies

There is only one exam in the class. This will be the final exam taking place on TBD.

Quizzes

There will be four short in-class quizzes. These will be conducted in your discussion sections in a timed, no notes, no devices format. Other details will be announced on Ed.

Grading

The course grade will be based on the following evaluation:

• Homework (0%)
• Participation (15%)
• Quizzes (20%)
• Final (65%)

Gating component: If your Final Exam is not of passing quality (C- or better), you cannot get higher than a D+ in the course. A high participation or quiz score cannot make up for a failing exam.

This class is not graded on a curve. We follow fixed grade bins, and your grade is determined on how well you do, not how well your peers do. Everyone can earn an A. Everyone can fail. The course staff sincerely wants ALL of you to succeed in this class.

Grade Overall Percentage A [90, 100] A- [88, 90) B+ [84, 88) B [75, 84) B- [68, 75) C+ [65, 68) C [62, 65) C- [58, 62) D [53, 58) F [0, 53) Individual categories of the grade can go negative. The grade of A+ will be awarded at the instructor’s discretion based on exceptional performance.

If you are taking the class PNP, you will need to attain a letter grade of C- or higher. If you are a graduate student taking the class SUS, you will need to attain a letter grade of B- or higher.

Participation

We expect students to participate in the class in a way that contributes to a positive and inclusive learning atmosphere for fellow students, and helps everyone deepen their mastery of the subject. We expect you to conduct yourself professionally in this course. Actions done in furtherance of these goals earn positive points. Bad citizenship and non-constructive behavior gets negative points. Doing nothing gets zero points. More details will be provided in a relevant Ed thread.

Everyone starts at zero points, which is a failing grade. In particular, you will be required as a part of participation and good citizenship to sign up for “special contributions” during the semester — although you can sign up for more to gain extra credit. These special contributions will both enhance your learning as well as those of your classmates. Other requirements for participation include participating in discussion and/or participating on Ed. Details will be in the Ed Thread.

Participation is made of two components: Ordinary participation (Two-thirds of the participation grade) Either: regular and full participation in the in-person discussion sections each week Or: active helpful participation on Ed: typically this is helpful participation in the homework problem threads, but can also be similarly helpful participation in discussion or lecture threads. Other possibilities also exist — but there is no spamming way to do this. Or: a combination of 1&2 above

1. Special participation (the remaining one-third) More information will be posted about this on Ed, but will involve various opportunities for interacting with the optimization material, highlighting contributions of other students, or supporting each other in the course.

DSP

We are happy to provide accommodations to ensure you can succeed in the class, please reach out as necessary.

Course Communication

The instructors and TAs will post announcements, clarifications, hints, etc. on Ed. You must check the EECS127/227A Ed page frequently throughout the term.

If you have a question, your best option is to post a message there. The staff will check the forum regularly, and other students will be able to help you too. When using the forum, please avoid off-topic discussions. Also, always look for a convenient category to post the question to (for example, each homework will have its own category, so please post there). That will ensure you get the answer faster.

If your question is personal or not of interest to other students, you may mark your question as private on Ed, so only the instructors will see it. If you wish to talk with one of us individually, you are also welcome to come to our office hours.

It can be challenging for the instructors to gauge how smoothly the class is going. We always welcome any feedback on what we could be doing better. You are encouraged to email us (ranade@eecs.berkeley.edu) with feedback at any time.

Collaboration

We encourage you to work on homework problems in study groups of two to four people; however, you must always write up the solutions on your own. Similarly, you may use books or online resources to help solve homework problems, but you must always credit all such sources in your writeup, and you may never copy material verbatim. Using previous homework and exam solutions is strictly prohibited, and will be considered academic dishonesty. This is not how you want to start your career as an engineer.

We expect that most students can distinguish between helping other students and cheating. Explaining the meaning of a question, discussing a way of approaching a solution, or collaboratively exploring how to solve a problem within your group is an interaction that we strongly encourage. But you should write your homework solution strictly by yourself so that your hands and eyes can help you internalize the subject matter. You should acknowledge everyone whom you have worked with, or who has given you any significant ideas about the homework. This is good scholarly conduct.

Don’t Be Afraid to Ask for Help

Are you struggling? Please come talk with us! The earlier we learn about your struggles, the more likely it is that we can help you. Waiting until right before an exam or the last few weeks of the semester to let us know about your problems is not an effective strategy - the later it is, the less we will be able to help you.

Even if you are convinced that you are the only person in the class who is struggling, please overcome any feelings of embarrassment or guilt, and come ask for help as soon as you need it – we can almost guarantee you’re not the only person who feels this way. Don’t hesitate to ask us for help – we really do care that you thrive!

Advice

The following tips are offered based on our experience.

Do the homework! The homework is explicitly designed to help you to learn the material as you go along. There is usually a strong correlation between homework scores and final grades in the class.

Keep up with lectures! Discussion sections, labs and homework all touch on portions of what we discuss in lecture. Students do much better if they stay on track with the course. That will also help you keep the pace with your homework and study group.

Take part in discussion sections! Discussion sections are not auxiliary lectures. They are an opportunity for interactive learning. The success of a discussion section depends largely on the willingness of students to participate actively in it. As with office hours, the better prepared you are for the discussion, the more you are likely to benefit from it.

Come to office hours! We love to talk to you and do a deep dive to help you understand the material better.

Form study groups! As stated above, you are encouraged to form small groups (two to four people) to work together on homework and on understanding the class material on a regular basis. In addition to being fun, this can save you a lot of time by generating ideas quickly and preventing you from getting hung up on some point or other. Of course, it is your responsibility to ensure that you contribute actively to the group; passive listening will likely not help you much. Also recall the caveat above, that you must write up your solutions on your own. We strongly advise you to spend some time on your own thinking about each problem before you meet with your study partners; this way, you will be in a position to compare ideas with your partners, and it will get you in practice for the exams. Make sure you work through all problems yourself, and that your final write-up is your own. Some groups try to split up the problems (“you do Problem 1, I’ll do Problem 2, then we’ll swap notes”); not only is this a punishable violation of our collaboration policies, it also ensures you will learn a lot less from this course.