Syllabus

Course Title: Optimization Models in Engineering

Units: 4

Course Format: 3 hours of lecture, 1 hour discussion per week

Contact Information:

• eecs127@berkeley.edu (emails visible to head TAs and professor).
• ranade@eecs.berkeley.edu (emails visible to professor).

Prerequisites: EE 16A, EE 16B, CS 70, Math 53

Online Resources: Ed, Gradescope, course website.

Course Description

This course offers an introduction to optimization models and their applications, ranging from machine learning and statistics to decision-making and control, with emphasis on numerically tractable problems, such as linear or constrained least-squares optimization.

Desired Course Outcome

By the end of the term, students having taken Optimization Models in Engineering are expected to be able to:

1. Understand the basic concepts of linear algebra: vectors, matrices, rank, projections; symmetric matrices, positive semidefinite matrices, eigenvalues; singular value decomposition and principal component analysis.
2. Understand three basic optimization models: least-squares, linear programming, and quadratic programming; and develop an understanding of the more general convex optimization.
3. Be aware of the wide-ranging applications where optimization models are useful, including in machine learning applications.
4. Be able to use prototyping software such as CVX to develop optimization-based solutions in concrete applications.

Course Content

The course content will roughly follow the outline posted on the website, but is subject to change and evolve throughout the semester.

1. Eigenvalue and singular value decomposition
2. Linear equations
3. Least squares
4. Convexity
5. Gradient descent
6. Linear programming
7. Quadratic programming
8. Duality
9. Optimality conditions
10. Applications: machine learning
11. Applications: control

Some application topics might be added mid-semester. It is strongly recommended that the students use the book Optimization Models, G.C. Calafiore and L. El Ghaoui, Cambridge University Press, October 2014.

Discussion

Discussion sections will take place every week, and are intended to provide further practice with the material and an opportunity to ask questions in a smaller class environment. You may attend any section. Discussions will begin on January 19th (the first week of classes). The discussions on a given Friday and the next Monday cover the same content. The discussion schedule is:

• Monday 10-11 AM, Dwinelle 229 (Staff: Aditya/Wayne)
• Monday 3-4 PM, Etcheverry 3107 (Staff: Aditya/Wayne)
• Monday 4-5 PM, Etcheverry 3107 (Staff: Aditya/Wayne)
• Friday 2-3 PM, Etcheverry 3107 (Staff: Druv/Sameer)
• Friday 3-4 PM, Etcheverry 3107 (Staff: Druv/Sameer)
• Friday 4-5:30 PM, Cory 540AB (Staff: Druv/Sameer)

Solutions to the discussion material will be released after the last discussion (on Monday).

Homework

Homework will be assigned every Friday on the course website and will be due the following Friday night at 11:00pm. Homework will be electronically submitted to Gradescope as a single PDF consisting of the following components:

1. Scanned handwritten work or a LaTeX file containing answers to non-IPython questions.
2. A PDF of all iPython notebooks.

You will need to select the pages associated with each question through Gradescope. Late homework will not be accepted. If you have extenuating circumstances (e.g., medical challenges) or DSP accomodations, you may submit a deadline extension request via the form at the top of this webpage prior to the deadline for the given homework assignment. We will drop the two lowest homework scores from the final grade calculation.

Discussion and collaboration on homework is encouraged, but direct plagiarism will not be tolerated. In other words, you are encouraged to discuss the homework with your classmates but you must write your own derivations and do your own calculations, etc. Do not hesitate to ask the professor or the GSIs for clarifications and hints for the homework problems during Homework Parties and Office Hours.

Homework Grading

The point of homework in this class is for you to learn the material. To help you in doing this each student will grade their own homework in addition to being graded by readers.

Self-Grades How-To

After the HW deadline, official solutions will be posted online. You will then be expected to read them and enter your own scores and comments for every part of every problem in the homework on a simple coarse scale.

• 0 = didn’t attempt or very very wrong,
• 2 = got started and made some progress, but went off in the wrong direction or with no clear direction,
• 5 = right direction and got half-way there,
• 8 = mostly right but a minor thing missing or wrong,
• 10 = 100% correct.

Note: You must justify self-grades of 2, 5 or 8 with a comment. Grades of 0 and 10 do not need to be justified. If you are really confused about how to grade a particular problem, you should post on Ed. This is not supposed to be a stressful process.

Each week, course readers will grade some (not necessarily strict) subset of the problems. These grades and comments serve as an opportunity to receive feedback on the work you’re submitting over the course of the semester. In general, your self-grades will count for your real homework grades; however, if we notice that your self-grades significantly differ from the reader grades, we will adjust your real grade to account for this difference.

Self grades will be released as Gradescope assignments after the homework due date. Your self-grades will be due on the following Friday after the homework deadline at 11:00 PM sharp. We will accept late self-grades up to 24 hours late (Saturday 11:00PM) for a 10% penalty on the associated homework assignment. If you don’t enter a proper grade by this deadline, you are giving yourself a zero on that assignment. Merely doing the homework is not enough, you must do the homework; turn it in on time; read the solutions; do the self-grade; and turn it in on time. Unless all of these steps are done, you will get a zero for that assignment.

Exam Policies

The 127/227AT Spring 2024 semester will have one midterm and one final. The midterm will be held on March 4, 2024, at 8-10 PM. The final will be held during the designated final exam slot released by campus. Makeup/alternate exams will not be scheduled under any circumstances.

Grading Breakdown

The course grade will be based on the following evaluation:

• Homework (25%)
• Midterm (35%)
• Final (40%)

There will also be a project option. If you choose to do the project, we will calculate the following breakdown:

• Homework (25%)
• Project (10%)
• Midterm (30%)
• Final (35%)

Your final grade will be the maximum of the two grading schemes.

Exam Policy: Exams cover all material until the exam. After the midterm, there will be a “midterm redo” assignment. If you complete the midterm redo, then you are eligible for a “midterm clobber”, i.e., your grade percentage on the midterm will be replaced with a weighted average of the midterm and final grade percentages if it is higher, using the following formula:

• clobbered midterm % = max(midterm %, 0.4 * midterm % + 0.6 * final %)

If you do not take an exam, you will get a zero for the exam, and will not qualify for the clobber policy.

DSP

We honor and respect the different learning needs of our students, and are committed to ensuring you have the resources you need to succeed in our class. If you need religious or disability-related accommodations, please let us know. If you requested accommodations, you should receive an acknowledgment email within the first few weeks of classes. Please email us if you have any questions.

We will be working with an external course manager to ensure that DSP accommodations are met.

Course Communication

The instructors and TAs will post announcements, clarifications, hints, etc. on Ed. You must check the EECS127/227A Ed page frequently throughout the term.

If you have a question, your best option is to post a message there. The staff will check the forum regularly, and other students will be able to help you too. When using the forum, please avoid off-topic discussions, and please do not post answers to homework questions before the homework is due. Also, always look for a convenient category to post the question to (for example, each homework will have its own category, so please post there). That will ensure you get the answer faster.

If your question is personal or not of interest to other students, you may mark your question as private on Ed, so only the instructors will see it. If you wish to talk with one of us individually, you are also welcome to come to our office hours. If your question relates to extenuating circumstances or other accommodation logistics, please send a message to the course email.

It can be challenging for the instructors to gauge how smoothly the class is going. We always welcome any feedback on what we could be doing better. You are encouraged to email us (eecs127@berkeley.edu) with feedback at any time.

Collaboration

We encourage you to work on homework problems in study groups of two to four people; however, you must always write up the solutions on your own. Similarly, you may use books or online resources to help solve homework problems, but you must always credit all such sources in your writeup, and you may never copy material verbatim. Using previous homework and exam solutions is strictly prohibited, and will be considered academic dishonesty. This is not how you want to start your career as an engineer.

We expect that most students can distinguish between helping other students and cheating. Explaining the meaning of a question, discussing a way of approaching a solution, or collaboratively exploring how to solve a problem within your group is an interaction that we strongly encourage. But you should write your homework solution strictly by yourself so that your hands and eyes can help you internalize the subject matter. You should acknowledge everyone whom you have worked with, or who has given you any significant ideas about the homework. This is good scholarly conduct.

Professionalism and Class Norms

We expect you to conduct yourself professionally in this course. To that end, we reserve the right to a grade penalty should you do anything inappropriate that compromises the well-being of your fellow students or staff. All students and staff in this course must be treated with respect by other members of the community.

Don’t Be Afraid to Ask for Help

Are you struggling? Please come talk with us! The earlier we learn about your struggles, the more likely it is that we can help you. Waiting until right before an exam or the last few weeks of the semester to let us know about your problems is not an effective strategy - the later it is, the less we will be able to help you.

Even if you are convinced that you are the only person in the class who is struggling, please overcome any feelings of embarrassment or guilt, and come ask for help as soon as you need it – we can almost guarantee you’re not the only person who feels this way. Don’t hesitate to ask us for help – we really do care that you thrive!

Advice

The following tips are offered based on our experience.

Do the homework! The homework is explicitly designed to help you to learn the material as you go along. There is usually a strong correlation between homework scores and final grades in the class.

Keep up with lectures! Discussion sections, labs and homework all touch on portions of what we discuss in lecture. Students do much better if they stay on track with the course. That will also help you keep the pace with your homework and study group.

Take part in discussion sections! Discussion sections are not auxiliary lectures. They are an opportunity for interactive learning. The success of a discussion section depends largely on the willingness of students to participate actively in it. As with office hours, the better prepared you are for the discussion, the more you are likely to benefit from it.

Come to office hours! We love to talk to you and do a deep dive to help you understand the material better.

Form study groups! As stated above, you are encouraged to form small groups (two to four people) to work together on homework and on understanding the class material on a regular basis. In addition to being fun, this can save you a lot of time by generating ideas quickly and preventing you from getting hung up on some point or other. Of course, it is your responsibility to ensure that you contribute actively to the group; passive listening will likely not help you much. Also recall the caveat above, that you must write up your solutions on your own. We strongly advise you to spend some time on your own thinking about each problem before you meet with your study partners; this way, you will be in a position to compare ideas with your partners, and it will get you in practice for the exams. Make sure you work through all problems yourself, and that your final write-up is your own. Some groups try to split up the problems (“you do Problem 1, I’ll do Problem 2, then we’ll swap notes”); not only is this a punishable violation of our collaboration policies, it also ensures you will learn a lot less from this course.