---

Lecture slides and references

The current list is based on a previous course in 2015. Slides and lecture orders will be updated and this page is updated periodically based on the pace of class presentation, project needs, and the feedback from you.

Weeks 1 and 2: Information Retrieval and Search Systems

• Introduction
• Overview of Information Retrieval and Web Search
• Information retrieval model.
• Open-source search engines and Lucene/Solr
• Search engine evaluations
• HW1

References:

• T. Yang, A. Gerasoulis, Web Search Engines: Practice and Experience . in Computer Science Handbook (T. Gonzalez. Eds), Chapman & Hall/CRC Press, 2014.
• Retrieval models, index construction/compression of textbook [MRS]
• Apache Lucene tutorial . Solr tutorial . Scaling Lucene/Solr
• Evaluation

Weeks 3 and 4: Ranking and Indexing

• Ranking and Learning
• Learning ensembles. Gradient tree boosting ( Slide set 1, Set 2)
• Project ideas
• Indexing

References:

• Computing scores , Learning to rank,
• Y. Freund et al. An efficient boosting algorithm for combining preferences, J of Machine Learning Research, 2003
• A. Mohan, Z. Chen, K. Weinberger, Wearch Ranking with Initialized Gradient Boosted Regression Trees Journal of MLR, 2011.
• Inverted files for text search engines by Justin Zobel, Alistair Moffat, ACM Computing Survey, Vol 38, No.2, 2006
• HW2

Weeks 5, 6, and 7: Offline Search Architecture and Text Mining

• Crawler in Search Engines
• Offline data processing
• MapReduce data processing
• Duplicate Detection and Similarity Computing
• Classification Algorithms. Deep learning with neural networks.

References:

• Crawling of textbook [MRS].
• MapReduce.
• Near duplicates from [MRS]
• Machine learning in automated text categorization by Fabrizio Sebastiani, ACM Computing Surveys, Vol 34 2002.
• Text classification in [MRS], SVM of textbook [MRS]
• Deep Feedforward Networks. Calculus on Computational Graphs. TensorFlow OSDI 2016 paper.

Weeks 8 and 9: Online Search Architecture and User Behaviors

• Query Processing Architecture and Online Systems
• Click Analysis
• Question Answering
• Recommendation
• Optional review problems Discussions

References:

• Neptune: Scalable Replica Management and Programming Support for Cluster-based Network Services. by K. Shen, et. al, 3rd USENIX Sympo. on Internet Technologies and Systems (USITS '01). ZooKeeper.
• Improving web search ranking by incorporating user behavior information by Eugene Agichtein, Eric Brill, Susan Dumais. SIGIR 2006.

Week 10: Review and Projects

Summary