Introduction
The Technology Department has a number of annual goals in support of the Wikimedia Foundation's Annual Plan; this work is detailed in the Annual Plan. Our remaining work falls into four broad areas—foundational, sustaining, supporting our technical community, and supporting the overall community health.
All Technology programs fall under the primary goal of Knowledge as a Service/Foundational Strength - evolve our systems and structures, except for TEC5: Scoring Platform and TEC9: Address Knowledge Gaps which fall under the primary goal of Knowledge Equity - grow new contributors and content.
Purpose of this document
Goals for the Wikimedia Technology department, for the second quarter of fiscal year 2018–19 (October - December 2018). The goal owner in each section is the person responsible for coordinating completion of the section, in partnership with the team(s) and relevant stakeholders.
Goals for the Audiences department are available on their own page
Legend
ETA (Estimated Time of Arrival) fields may use the acronym EOQ (End of Quarter) or EOY (End of Year).
Status fields can use the following templates:  To do,
 To do,  In progress,
 In progress,  Blocked,
 Blocked,  Postponed,
 Postponed,  Stalled,
 Stalled,  Partially done, or
 Partially done, or  Done
 Done
Technology Departmental programs
TEC1: Reliability, Performance, and Maintenance
Goal Owners: Mark Bergsma; Ian Marlier; Nuria Ruiz; Bryan Davis
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2 by team:
Analytics
- Continue upgrading to Debian Stretch  Done Done
- Order and configure hardware for dbstore1002's replacement  Done Done
- Add prometheus metrics for varnishkafka task T196066  In progress In progress
- Working on a strategy and scripts for updating superset (http://superset.wikimedia.org) task T211706  In progress In progress
SRE
- Refresh hardware and perform necessary maintenance - will be  Done by end of December Done by end of December
SRE / Traffic
- ATS production-ready as a backend cache layer will be  Done by end of December Done by end of December
- Migrate most standard public TLS certificates to CertCentral issuance  Done Done
- Increase Network Capacity  Partially done, will be completed in Q3 due to ongoing fundraising efforts Partially done, will be completed in Q3 due to ongoing fundraising efforts
RelEng
- Determine the procedure and requirements for an automated MediaWiki branch cut  Done Done
Performance
- Train feature developers on the use of performance metrics to detect and address regressions  In progress In progress
- Deliver high-traffic images as WebP  Done Done
- Improve Navigation Timing data, by moving it from Graphite to Prometheus  Partially done Partially done
- Expand mobile testing  In progress In progress
- Expand outreach and engagement to wider Performance community  In progress In progress
- Test the effect of MediaWiki commits  In progress In progress
- Ongoing maintenance of components owned by Performance is always  In progress; for this quarter's work, we're In progress; for this quarter's work, we're Done Done
- Anonymized data publishing is  Stalled and deferred to Q3 Stalled and deferred to Q3
- Research performance perception in order to identify specific metrics that influence user behavior  Done with follow-up in Q3 Done with follow-up in Q3
WMCS
- Continue replacing Trusty with Debian Jessie/Stretch in infrastructure layer  In progress and will continue in Q3 In progress and will continue in Q3
- Communicate Trusty deprecation to Cloud VPS community  Done Done
- Develop Trusty deprecation plan for Toolforge  Done and communicate that timeline to community Done and communicate that timeline to community Partially done Partially done
- Track progress towards full removal of Trusty from Cloud VPS  In progress, will continue in Q3 In progress, will continue in Q3
- Migrate 50% of Cloud VPS projects to the eqiad1 region and its Neutron SDN layer  Done, will continue in Q3 Done, will continue in Q3
TEC2: Modern Event Platform
Goal Owner: Nuria Ruiz
Q2 Goals are  Done
 Done
- Detailed status here.
Wrap-up for Q2:
- Development of intake service for events whose transport is JSONSchema/http  Done Done
TEC3: Deployment Pipeline
- Goal Owner: Greg Grossmeier
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Formalize the collection of CI infrastructure and tooling metrics is  Partially done will continue in Q3 to expose the interface metrics Partially done will continue in Q3 to expose the interface metrics
- Develop set of metrics to assess incident reports/post mortems is  Done, will probably do follow-up work in Q3 Done, will probably do follow-up work in Q3
- Adopt more services into Deployment pipeline - migrate graphoid to the Deployment pipeline is  Postponed as Graphoid is now recommended for stewardship review, zotero v2 is Postponed as Graphoid is now recommended for stewardship review, zotero v2 is Done Done
- Deploy blubberoid  Done Done
- Reprise the work on the logging infrastructure  In progress and will continue in Q3 In progress and will continue in Q3
TEC4: PHP7 Migration
- Goal Owners: Mark Bergsma and Ian Marlier
Q2 Goals are  Done
 Done
- Detailed status here.
Wrap-up for Q2:
- Ability to serve a % of production traffic from PHP7 are mostly  Done with final code reviews Done with final code reviews In progress In progress
- Sampling profiler for PHP7 has been identified and is prepared for use in the WMF production environment  Done Done
- Identify and address code issues and opportunities under PHP 7.2  Done Done
TEC5: Scoring Platform
- Goal Owner: Aaron Halfaker
Q2 Goals are  Done
 Done
- Detailed status here.
Wrap-up for Q2:
- Upgrade Celery  Done Done
- Fix logging for logstash  Done Done
- Implement edit quality models for translatewiki   Partially done and will wrap up by end of December Partially done and will wrap up by end of December
- Document Feature Injection in The ORES Manual  Done Done
- Blog announcement of Undisclosed Paid Editors dataset  Done Done
- Resubmit ORES paper to the Journal of Computing  Cancelled Cancelled
- JADE --> Production  Partially done to be finished up in Q3 Partially done to be finished up in Q3
TEC6 Address Infrastructure Gaps
- Goal Owner: Mark Bergsma
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Begin the implementation of Q1's Logging Infrastructure design is mostly complete and will be  Done by end of December Done by end of December
- Expand modern metrics infrastructure coverage is mostly complete and will be  Done by end of December Done by end of December
- Design and prepare infrastructure for database binary backups is  In progress and will continue in Q3 In progress and will continue in Q3
- Test Performance implications of MySQL TLS connectivity  Stalled on DBA technology selection/implementation due to other work requirements that have higher priorities Stalled on DBA technology selection/implementation due to other work requirements that have higher priorities
- Start migrating watchlist last-view updates is  Stalled due to emergent work and other higher priority work, we hope to get it done in early Q3 Stalled due to emergent work and other higher priority work, we hope to get it done in early Q3
- Expand Spicerack library and SRE Cookbooks conversion is  Partially done and will continue into Q3 Partially done and will continue into Q3
- Expand Netbox usage  Done with stretch goals to be done in Q3 Done with stretch goals to be done in Q3
TEC7: Environmental Sustainability
- Goal Owner: Erika Bjune
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Identify and contract with an organization that can assess WMF's environmental footprint is  Done Done
- Work on an actionable plan for reducing WMF's environmental footprint is now  In progress and will be on-going for ~6 months. In progress and will be on-going for ~6 months.
TEC8: Search Platform
- Goal Owner: Erika Bjune
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Find and hire a contractor to help with NLP work  Done Done
- Begin working on one internal NLP project  Partially done Partially done
- Improve autocomplete of Wikidata items  Partially done and will continue in Q3 Partially done and will continue in Q3
- Prototype a feature that is based on collected data  Partially done and will continue in Q3 Partially done and will continue in Q3
- Finish up the Korean morphological library analysis and get ready for deploy into production when ES6 is completed  Done Done
- General language support is always  In progress In progress
- Search for licenses in Commons is  Stalled as we await further instructions from SDoC program Stalled as we await further instructions from SDoC program
- Split the search clusters to increase stability  Done Done
- Continue replacing ElasticSearch servers (end of life maintenance)  Done Done
- Separate the Wikidata ElasticSearch implementation into a separate extension  Partially done and will continue in Q3 Partially done and will continue in Q3
- Migrate ElasticSearch cluster restart scripts as cookbooks using Spicerack  In progress as more testing is needed In progress as more testing is needed
- Performance and bug fixes for WDQS is always  In progress In progress
- Service Level Objective (SLO) work for WDQS is  In progress and will continue in Q3 In progress and will continue in Q3
- Investigate Blazegraph support options and alternatives  Done Done
TEC9: Address Knowledge Gaps
- Goal Owner: Leila Zia
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Iterate on and improve the report of the state of the art on bias detection and algorithm audibility is  Postponed Postponed
- Build a section recommender system based on the section mapping algorithm is  Partially done and will be finished by end of December Partially done and will be finished by end of December
- Build a test API for the section recommendation algorithm is  Postponed until Q3 Postponed until Q3
- Improve article recommendation API to completion (of the second stage improvements) is  Partially done and will be finished by end of December Partially done and will be finished by end of December
- Expand the taxonomy of Wikipedia readers is  Done Done
- Preparing the infrastructure for conducting the survey is  Partially done and will be finished by end of December Partially done and will be finished by end of December
- Devise the framework for matching newcomers to improve the first design of the framework  Partially done and will be continued in Q3 Partially done and will be continued in Q3
- Develop and test a new experiment plan for testing the quality of the algorithm to elicit user interests  Done Done
- Finalize the documentation for the research on characterizing Wikipedia readers  Done Done
- A series of presentations about the results on characterizing Wikipedia readers  Done Done
TEC10: Build Technical Community
- Goal Owner: Bryan Davis
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Plan and visibly improve Toolforge technical documentation is  Partially done and will continue in Q3 Partially done and will continue in Q3
- Survey Wikimedia Foundation staff to gauge interest and support for reviving Tech Talks is  Done Done
- Develop plan for Tech Talks reboot is  In progress and will continue in Q3 In progress and will continue in Q3
- Update visual design and content of MediaWiki.org Main Page will be  Partially done by end of January 2019 Partially done by end of January 2019
- Support Outreachy Round 17  Done Done
- Support Google Code-In 2018  Done Done
- Review and improve top viewed overview pages of the Action API  Done Done
- Submit a proposal for the Wiki Research Workshop  Done Done
TEC11: Support Fundraising Activities
- Goal Owner: Erika Bjune
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Support Advancement in all Q2 activities  Partially done and will continue in Q3 Partially done and will continue in Q3
TEC12: Developer Productivity
- Goal Owner: Greg Grossmeier
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- The Annual Developer Productivity Survey results are synthesized and shared, creating a first year baseline is  Partially done and will continue in Q3 to get additional feedback Partially done and will continue in Q3 to get additional feedback
TEC13: Code Health
- Goal Owner: Greg Grossmeier
Q2 Goals are  In progress
 In progress
- Detailed status here.
Wrap-up for Q2:
- Update/refresh review queue is  Partially done and will continue in Q3 Partially done and will continue in Q3
- 5 of the 15 prioritized repositories have at least 1 end-to-end test is  In progress as the team talks with stakeholders In progress as the team talks with stakeholders
- Assess Platform unit test practices and define improvement plan is  In progress and will continue in Q3 In progress and will continue in Q3
- Core Platform and Search Platform teams are using TDM PoC  In progress and will continue in Q3 In progress and will continue in Q3
- Identify key Tech Debt areas and add a process for management  In progress and will continue in Q3 In progress and will continue in Q3
- Metrics defined and deployed for all 4 Code Health areas  Partially done Partially done
TEC14: Smart Tools for Better Data
- Goal Owner: Nuria Ruiz
Q2 Goals are  Partially done
 Partially done
- Detailed status here.
Wrap-up for Q2:
- Create report for "articles with most contributors" in Wikistats2  To do To do
- Create report for Active editor metrics per project family  To do To do
- Provide easier mapping between Wikistats1 metrics and Wikistats2 metrics  To do To do
- Provide ability to query metrics per project family in Wikistats2  Done Done
- Add per family unique devices to analytics query service  Done Done
- Automatic ingestion from eventlogging data into turnilo datasets that area available for easy exploration  Done Done
- Automation of data sanitization for eventlogging schemas in the hadoop backend  Done Done
- Presto cluster online and infrastructure accessible by Cloud (labs) users  Done Done
- Edit Data Lake Quality - resolve known issues (ongoing)  In progress In progress
- POC More efficient Bot filtering on pageview data  Done Done
- Productionize MediaWiki content processing, ngest and process XML dumps  Done Done
Cross-departmental programs
CDP1: Privacy, Security, and Data Management
Segment 2 - Security
- Goal Owner: John Bennett
Q2 Goals are  In progress
 In progress
- Detailed status here.
Wrap-up for Q2:
- Review and mature our security policies and awareness functions is  Done but the phishing campaign is Done but the phishing campaign is Stalled to be completed in Q3 Stalled to be completed in Q3
- Testing campaigns:
- CSP changes are now  Done Done
- 1st round of pen testing (on en wikipedia) is  Done Done
- OIT assessment is  Cancelled, might be picked up in 2019. Cancelled, might be picked up in 2019.
- NIST CSF assessment is  Stalled, should be picked up again in early 2019. Stalled, should be picked up again in early 2019.
- Initial discussion is  In progress to include Phan into MW core and should be completed by end of December. In progress to include Phan into MW core and should be completed by end of December.
 
- CSP changes are now 
- Finalize and test our Incident Response documentation is  In progress and will continue in Q3 In progress and will continue in Q3
Segment 3 - Analytics
Goal Owner: NRuiz (WMF)
Wrap-up for Q2:
- More restrictive Firewall rules for Kafka. task T204957  Postponed Postponed
- Review the requirements for a service implementing a stronger user authentication scheme for the Analytics Hadoop cluster and possibly for other related tools (like Zookeeper).  Done Done
- STRETCH GOAL: implement a prototype in labs that the Analytics team can test and evaluate. task T198227  Done Done
 
- More restrictive Firewall rules for Kafka. task T204957 
CDP2: Platform Evolution
Segment 7 - Core Platform
Goal Owner: Corey Floyd (WMF)
Q2 Goals are  In progress
 In progress
- Detailed status here.
Wrap-up for Q2:
- Define and implement a session management service  Done Done
Segment 8 - Core Platform (WMDE)
Goal Owner: Corey Floyd (WMF)
Q2 Goals are  In progress
 In progress
- Detailed status here.
Wrap-up for Q2:
- Wikimedia Technical Conference: participate and analyze session output is  Done and will be published soon Done and will be published soon
CDP3: Knowledge Integrity
Segment 1 - Research
Goal Owner: Dario Taraborelli
Q2 Goals are < In progress
 In progress
- Detailed status here.
Wrap-up for Q2:
- Design a machine learning framework to identify why statements need a citation in English Wikipedia  Done Done
- Submit a paper summarizing the modeling work for unsourced statement detection  Done Done
- Run the second round of data collection to understand Wikipedia citation usage  Done Done
- Prepare the data and analyze the data collected in the second round  Done Done
- Perform first round of survey data collection of reader citation usage on English Wikipedia  Done Done
- Analyze first round survey data of reader citation usage  In progress and will continue in Q3 In progress and will continue in Q3
- Host WikiCite 2018 event  Done Done
CDP4: Structured data
Segment 2 - Search Platform
Goal Owner: Erika Bjune
Q2 Goals are  In progress
 In progress
- Detailed status here.
Wrap-up for Q2:
- Allow search by type of license  Stalled until Q3 Stalled until Q3

