Security and Privacy of Machine Learning

— 1 Jan 0001

Teams

Responsibilities

For each week (except for project proposal and presentation weeks), one team will be responsible for Leading the class, one team for writing a Blog post on the class topic, and one team for arranging food. See the Schedule for team responsibilities.

Leading Team. The team responsible for leading a class should:

• Two weeks before the scheduled class, meet briefly with me (Dave) to discuss plan for the class. You should decide on a team leader for this class, who will be the one responsible for making sure everyone on the team knows what they are doing and coordinating the team’s efforts.

• The Monday the week of the class, at least a few representatives from the team should come to my office hours to discuss the plan for the class. You should come prepared to this meeting with suggested papers and ideas about how to present them.

• No later than the Monday before class, send out to the class Slack group the preparation materials for the class. This can include links to papers to read, but could also include exercises to do or software to install and experiment with, etc.

• Day of class: lead an interesting, engaging, and illuminating class! This is a 2.5 hour class, so it can’t just be a series of unconnected, dull presentations. You need to think of things to do in class to make it more worthwhile and engaging.

• After class: help the Blogging team by providing them with your materials, answering their questions, and reviewing their write-up.

Blogging Team. The team responsible for blogging a class should:

• The week before the scheduled class, develop a team plan for how to manage the blogging. One team member should be designated the team leader for the blogging, and post on slack so we know who is responsible. The blogging leader is responsible for making sure the team is well coordinated and everyone knows what they are doing and follows through on this.

• During class, participate actively in the class, and take detailed notes (this can be distributed among the team).

• By the Tuesday following class, have a draft of the blog post ready, and share it with the rest of the class (including the leading team and coordinators) for comments. Details on how to prepare the blog post are on the Blogging Mechanics page.

• By the next Friday (one week after the class), have a final version of the blog post ready to add to the course site.

Feeding Team. The team responsible for food should:

• Plan among yourselves what food to bring and who is responsible. If you want to use my credit card to buy food, borrow it. You can stop by Thursday afternoon to pick it up from me.

• Bring something yummy and something with caffiene, but not too messy or disruptive, to class. A simple choice is to get a coffee container and bagels from the bagel shop in Rice Hall (but make sure to get the order in early enough to be ready before class). More adventurous choices are encouraged.

• Make sure to clean up the room at the end of class. If we get caught leaving a mess, we probably will not be allowed to have food anymore.

• Save the receipts to get reimbursed, and take care of the reimbursement. This is easiest if you just borrow my credit card and then all you need to do is send me an image of the receipt (or hand me a physical receipt).

— 1 Jan 0001

This page collects some topic ideas and papers for future classes. These are just suggestions, not meant to be an exhaustive list or limit the scope of future topics or papers.

Certified Defenses, Formal Methods

Aditi Raghunathan, Jacob Steinhardt, Percy Liang. Certified Defenses against Adversarial Examples. PDF

Guy Katz, Clark Barrett, David Dill, Kyle Julian and Mykel Kochenderfer. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. PDF

Nicholas Carlini, Guy Katz, Clark Barrett, and David L. Dill. Ground-Truth Adversarial Examples. PDF

J. Zico Kolter, Eric Wong. Provable defenses against adversarial examples via the convex outer adversarial polytope. PDF

Testing

Kexin Pei, Yinzhi Cao, Junfeng Yang, Suman Jana. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. PDF

Defining Adversarial Examples

Mahmood Sharif, Lujo Bauer, Michael K. Reiter. On the Suitability of \(L_p\)-norms for Creating and Preventing Adversarial Examples. arxiv, Feb 2018. PDF

Gamaleldin F. Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian Goodfellow, Jascha Sohl-Dickstein. Adversarial Examples that Fool both Human and Computer Vision. arxiv, Feb 2018. PDF

Adversarial Training

Alex Kurakin, Dan Boneh, Florian Tramèr, Ian Goodfellow, Nicolas Papernot, Patrick McDaniel. Ensemble Adversarial Training: Attacks and Defenses. ICLR 2018. Web

Malware Detection and Evasion

Robin Sommer and Vern Paxson. Outside the Closed World: On Using Machine Learning For Network Intrusion Detection. PDF

Igino Corona and Giorgio Giacinto and Fabio Roli. Adversarial Attacks against Intrusion Detection Systems: Taxonomy, Solutions and Open Issues. PDF

Kyle Soska and Nicolas Christin. Automatically Detecting Vulnerable Websites Before They Turn Malicious. PDF

Roberto Jordaney, Kumar Sharad, Santanu K. Dash, Zhi Wang, Davide Papini, Ilia Nouretdinov, and Lorenzo Cavallaro. Transcend: Detecting Concept Drift in Malware Classification Models. Paper Site

Hung Dang, Yue Huang, and Ee-Chien Chang. Evading Classifiers by Morphing in the Dark. PDF

Fairness

Aylin Caliskan, Joanna J. Bryson, Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. HTML

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. PDF

Conference on Fairness, Accountability, and Transparency

Assessing Risks

The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation. February 2018 PDF

Poisoning

Battista Biggio, Igino Corona, Giorgio Fumera, Giorgio Giacinto, and Fabio Roli. Bagging Classifiers for Fighting Poisoning Attacks in Adversarial Classification Tasks. PDF

Scott Alfeld, Xiaojin Zhu, and Paul Barford. Data Poisoning Attacks against Autoregressive Models. PDF

Blogging Mechanics
David Evans — 1 Jan 0001

Here are some suggestions for how to create the class blog posts for your assigned classes. I believe each team has at least a few members with enough experience using git and web contruction tools that following these instructions won’t be a big burden, but if you have other ways you want to build your blog page for a topic let me know and we can discuss alternative options.

• Install Hugo. Hugo is a static website generator that builds a site from Markdown pages. (With homebrew on Mac OS X, this is easy: brew update && brew install hugo.)

• Clone the github repository, https://github.com/secML/secML.github.io. This is what is used to build the https://secml.github.io/ site. If you are working with multiple teammates on the blog post (which you probably should be), you can add write permissions for everyone to the cloned repository.

• You should create your page in the web/content/post/ subdirectory. You can start by copying an earlier file in that directory (e.g., class1.md) and updating the header section (between the +++ marks) and replacing everything after that with your content. Don’t forget to update the date so your page will appear in the right order. You can put draft = true in the header, so your page will not appear on the course website until it is ready.

• You can use multiple files (but probably only one in the post/ directory (this will show up as pages on the front list). Use the web/content/images directory for images and the web/content/docs directory for papers. Using images and other resources to make your post interesting and visually compelling is highly encouraged!

• Write the blog page using Markdown. Markdown is a simple markup language that can be used to easily generate both HTML and other output document formats. You can probably figure out everything you need by looking at previous posts, but for a summary of Markdown, see Markdown: Syntax.

• You can incorporate latex math into your markdown. Use \\( inline \\) for inline math, e.g., \\( J_{min}\\} and $$ ... $$ for display math.

• Your post should include credits for any external material you use, especially for any images you incorporate that you didn’t produce.

• Please include full references to the papers, and links to the most definitive source available (usually this is to arxiv or a conference site, but could be the author’s page). You should include additional links to relevant and useful reference or code repositories. Its good to have an overview section at the beginning of the post with links to all the main papers covered, and then to have links in specific sections to what is being covered.

• To test the post, run make develop (in the web/ subdirectory of your repository). This starts the Hugo development server, usually on port 1313 (unless that port is already in use). Then, you can view the site with a browser at localhost:1313.

• When you are ready, submit a pull request to incorporate your changes into the main repository (and public course website). Also, send a message to me (dave) on slack, so I know the post is ready to review. At this stage, I will probably make some requests for improvements, and then will post the edited version to the course site.