DEVIEW 2021

Contents List

1. Introduction

• What is Clova Face Kit?
• supported features
• Used in:
  • GangnamUnni
  • FaceSign (will be installed in our new office)
  • Naver photo cloud
  • Line eKYC

2. Hands-On

• Settings & Options
• iOS
• Android
• Python
• JavaScript

3. Closing

Brief

On-Device machine learning is spreading out beyond mobile platforms like iOS/Android.
The Clova Face Kit is a lightweight vision engine that supports mobile devices, Python, and web browsers.
We are pleased to present this hands-on to broaden users of Clova Face Kit - which is used in Naver services.

Will be helpful to:

• Everyone who is interested in On-Device ML.
• Everyone who is planing/developing a face recognition service.
• Everyone who wants to use face recognition feature in their own business.

In this session, we will share our experience of operating HBase-based data storage.

NAVER data storage is operated based on HBase while maintaining high HBase Locality.
Learn about HBase Locality and look at its internal structure to see how it can bring performance benefits. (HDFS Short-circuit Local Reads)
We will share how to operate while maintaining HBase Locality high and the problems and solutions we encountered while dealing with disk failures that lower HBase Locality.
Finally, we check the effect of using HDFS Short-circuit Local Reads on the datastore.

Contents

1. HBase-based data storage

• Why Naver needs data storage
• HBase-based search data storage (Deview 2017)
• Basic structure of HBase

2. What is HBase Locality?

• Data Locality
• Hbase Locality
• How to increase HBase Locality

3. Increase read performance through HBase Locality

• Order of LocatedBlocks
• HDFS Short-Circuit Local Reads

4. Operation to keep HBase Locality in NAVER data storage

• How to keep HBase Locality in operation
• Disk failure proportional to cluster size
• DataNode Hot Swap Drive
• Unmountable phenomenon 1
• Share state when HDFS block is broken
• Aggressive Purge
• Unmountable phenomenon 2
• Automate disk failure handling

5. Effect in Naver Data Store

Target audience

• Those who were interested in announcing the HBase-based search data store
• Those who are operating HBase-based large-capacity clusters
• Those who want to get performance benefits from HDFS-based data processing
• Those who are curious about the internal structure of HDFS shortCircuit Reads
• Those who are tired of handling disk failures when operating a large-scale cluster

In this session, we would like to share the experience of developing NAVER CLOUD PLATFORM Terraform provider.

Context

1. Terraform provider

• Prologue
• Terrform provider is?

2. How terraform works

• Learn how Terraform works through examples.

3. Terraform plugin

• Terraform plugin is?
• NCLOUD GO SDK
• Terraform plugin
• Implementation of provider
• Implementation of Resource
• Testing and Release

4. Appendix & Tips

• Handling async api in terraform
• Concurrency issue
• Circular cycle issue
• Tips

Contents

Homomorphic Encryption and Data Privacy

• Why is PPML necessary?
• Introduction to Homomorphic Encryption
• What does HE-based ML secure
• Issues when developing ML algorithms with HE

Computations in PPML Training, Inference

• Practical issues when dealing with HE-based ML code
• HE-based PPML open-source projects
• Designing computation circuits and HE cost model
• Optimizing approximated non-linear polynomials
• Optimizing matrix-vector multiplications used in CNN

Introduction to PPML Project Desilo has completed

• iDash 2020 (Secure multi-label Tumor classification)

Audience

• Software engineers interested in Privacy-preserving Machine Learning
• Software engineers interested in learning about modern cryptography and its applications in ML

We discuss how robots can learn to perform new tasks given only a few demonstrations of those tasks. In the past, people have addressed such "few-shot imitation problems", by training agents to copy the demonstrators' behaviours or by inferring reward functions from demonstrations. We present a new approach to few-shot imitation, called "demonstration-conditioned reinforcement learning (DCRL)" which has multiple advantages over past approaches: it can naturally address situations where the demonstrator and the agent are physically different (e.g. human demonstrator and robot agent); it can improve upon suboptimal demonstrations; and it can gracefully recover from unexpected situations (e.g. when an object slips out of the robot's grasp) even though such situations are never encountered in the demonstrations.

Table of Contents

1. Overview

• Explain the challenge of getting robots to perform diverse tasks and say why few-shot imitation is a promising way to address this challenge.

2. Few-Shot Imitation

• Formulate the few-shot imitation problem, discuss pre-existing approaches and closely-related problems.

3. Demonstration-conditioned reinforcement learning (DCRL)

• Describe the core principles of our new approach and the novel features of our transformer network architecture.

4. Benchmarking few-shot imitation performance

• Describe a robotic manipulation benchmark and a "blind navigation" benchmark.
• Illustrate the performance of our new approach relative to previously proposed baselines.

5. Outlook

• Discuss the main interesting directions for improving on our method and making it a practicable approach to everyday human-robot interaction.

Target

• Developers who are curious about robotics.

• Developers who are curious about reinforcement learning.

• Developers who would like to hear about applications of transformers outside of natural language processing.

Index

1. Search service and user

• Components of our search service
• How can we trace user behaviors?
• Search Log Analysis Dept

2. User behavior modeling

• User behavior modeling
• Primitive Feature
• Click Model?
• PGM Click Models
• Advanced PGM Click Models
• Neural Click Models
• Click Model Evaluation
• Future work

3. NAVER Dataset CLARA and Pyclick framework

• About click dataset
• Understanding click datasets through CLARA
• Newer version of CLARA: CLARA2
• Creation process of CLARA2
• Pyclick; Let's make our own Click Model
• Learning process using Pyclick
• Our Contribution
• Pyclick Tutorial

4. Apply to search service

• User Modeling Process

Target

• For those who develop search services such as shopping, images, videos, and web documents
• For those who don't know how to use the user behavior log
• For those who need dataset and framework to develop a user model
• For those who want to know the process of improving the search system with the user behavior log
• For those who want to learn Pyclick models

The 2020 Tokyo Olympics drew everyone's attention.
The background of the successful live broadcast of the Naver Olympics was the NAVER LIVE CLOUD team.
We would like to share the technology and experience that accommodated 1.4 million users.

Contents

1. 2020 Tokyo Olympics Live streaming

• 2020 Tokyo Olympics Live streaming
• The characteristics of live streaming service.
• The key point for successfully finishing the Olympic broadcast.

2. NAVER LIVE CLOUD Structure

• LIVE CLOUD Transcoder module Farm
• LIVE CLOUD Operator server

3. Traffic control - Operation experience, data, tuning, successful.

• Concerns and solutions to CDN traffic control.
• Full preparation for API traffic.

4. For the next Olympics

• Global live data transmission platform
• Beyond Live

Target

• Backend developers who are interested in controlling large traffic.
• Developer who is interested in live streaming.
• Developer who is interested in CLOUD system.

In this session, we will share the experience of building an AI platform based on Kubeflow, improving the model serving process, and MLOps.

Explains the reason for moving from the existing Hadoop-based AI platform such as C3DL and AiSP, which are already actively used by in-house developers, to the new Kubeflow-based AI platform.

After that, from the perspective of MLOps, we will share how we were able to connect with our Hadoop-based in-house platforms in the process and how we can make automated ml end-to-end pipeline (data processing, model learning, and serving) using Kubeflow Pipelines.

Lastly, we would like to share our experiences in building a Kubeflow-based AI platform. In this process, I would like to talk about the experience of introducing Alluxio in order to use HDFS as a volume.

Contents

1. Why Kubeflow-based AI platform?

• Hadoop-based in-house AI platform and limitations
• Advantages of Kubeflow-based AI platform

2. Serving large-scale ML services with KFServing

• Comparison with existing serving platforms
• Establishing the KFServing environment

3. Realizing MLOps with Kubeflow

• Introduction to Kubeflow components
• ML pipeline configuration by linking in-house platform

4. Multi-Tenant Kubeflow

• Multi-Tenancy support for Kubeflow
• Dedicated node pool support
• Ease of operation: Declaratively manage user-related policies

5. Alluxio integration to use HDFS as k8s volume

• Using Alluxio on k8s

6. Future work

• For better MLOps?
• For a better platform?

Target

• Those who are curious about Kubeflow, an ML platform based on Kubernetes
• Those who are interested in operating services by serving ML models
• Those who want to automate the ML process
• Those who want to build and operate Kubeflow multi-tenant
• Those who want to use HDFS as a volume in Kubernetes

We share build automation and project management know-how using Chat-Ops and Dev-Ops.
We experienced performance improvement by minimizing manpower.
It's about how app/service development teams will be free from non-development tasks and forcus more on development tasks and managing them efficiently.

It deals with the design and implementation of environments in tools and languages such as Slack / Jenkins / NELO / Fastlane / Python.

Audience

• All mobile app development team
• Anyone who is concerned about automating project management and minimizing the cost of non-development work.
• Anyone who wants to expand their capabilities besides app development technology
• Anyone who wants to improve their work efficiently in a remote work environment

Contents

1. Dev-Ops and messenger.

• Tools to build Dev-Ops and messengers.
• Build a CI with Jenkins.
• Build a CD with Jenkins and Fastlane.

2. Build Chat-Ops & Dev-Ops infrastructure

• Setup Dev-Ops.
• Setup Chat-Ops.
• Chat-Ops development.
• Chat-Ops usage.

3. Apply error alarm and upgrade with AI/ML.

• Collect NELO logs.
• Notification using Lambda
• Time-series forecasting

4. Epilogue

• Job Automation
• Trouble Shooting

Kubernetes application is expanding the scope of Stateful Container from Stateless Container.
For stateful containers, data needs to be stored “fast, secure and cost-effectively”.
For this, Naver is using Ceph, OpenStack, and Kubernetes through self-developed plugins.
Introducing how NAVER stores container data quickly, safely, and cost-effectively, and how to efficiently monitor it.

Contents

1. Stateful Application

• Stateful Application Containerizing
• Local vs Remote Storage
• Thick vs Thin Storage

2. Cloud Native Storage

• Kubenetes Storage Provisioner
• Custom Provisioner and Flow

3. Troubleshooting

• Performance Issue
• Distribution Application on Distribution Storage Issue
• Storage Scale-out Issue

4. Operation Tools and Tips

• Performance Monitoring with Thanos
• Log Monitoring with PLG Stack
• Grafana Dashboard Templates

Target Audience

• Developers who need to use Kubernetes Stateful Services
• Developers who need to use Network Storage in Kubernetes

• Introduction of Naver Place search service
• Introduction of data refinement
• Limitations and improvement points of the existing operation method
• SRE application plan for data refinement system
• Introduction of SRE platform COSMOS
• COSMOS features and advantages
  • Service failure prediction and defense
  • WitheBox Monitoring
  • Indicator-based service operation
  • System Architecture
• Examples of service application

Today, most businesses are moving based on data.
The amount and scope of data use are expanding endlessly.
At the same time, data-related regulations and laws are becoming more strengthened and complex.
The situation like that the number and use of data increases while regulations related to data also increase significantly makes
data users have less usability of data, and Infosec managers are overloading their work.

In this presentation, we share the implementation of Data Platform that increases user usability while using data safely and legally in the current situation.

About the lecture.

Starting with a basic explanation of SRE/DevOps for service quality, this lecture shares a number of know-how and tips based on the experiences we have gained by actually applying them within the service. We will look at how a well-formed development culture can affect service quality and developer life, and share how Naver actually develops/operates services under a remote environment.

We will provide helpful information from a service operation perspective, such as how to detect and cope with errors in various user environments, how to monitor services, and how to respond internally to the surge in service traffic with various events.

In addition, we will not just introduce the SRE/DevOps tool, but will basically inform you based on recent experiences in terms of service platform design structure/modularization for service quality and various DevOps introduction to support it.

Table of contents.

1. Technology and culture that strengthens the service.

• The sense of duty of the front line service developer.
• Devops in the non-face-to-face era

2. SRE, let's start like this!

• Don't check with your hands anymore.
• Let's verify whatever it is.
• Failure. If you can't avoid it, slow it down.
• Smart collaboration.

3. Try more. SRE!

• Automate service traffic detection and response.
• The bot checks the issue?
• A way to verify the content created by the creator.

Naver mobile live streaming has naturally become an essential technology to its successful service and also a representative function of the service. The mobile live streaming has been used for entertainment business (like V-Live), online community business (like BAND), and live commerce business (like Shopping Live), blending nicely with various kinds of other services.

The technology for publishing videos (production) and making their playback (consumption) is going to be introduced in this session.

Contents

1. Naver Mobile Live Streaming

• Our Most Important Live Services
• From Entertainment Business to Live Commerce Business
• The Status of Live Streaming in Everyday Life

2. Naver Mobile Technology of Publishing Live Videos

• Our Main Services Using the Technology of Publishing Live Videos
• The Importance of Standard SDK in NAVER
• The Limitations of RTMP and Introduction of a New Protocol
• Global and Mass Traffic Caused by COVID-19
• The improvement of Adaptive Bitrate Publish
• The Visual Technology Enabling Live Commerce to be more professional
• Two-Way Networking Technology for the Future

3. Naver Mobile Technology of Making Live Playback

• NAVER Cloud and Player SDK
• The Importance of Standard SDK in NAVER
• Various Live Scenarios Caused by COVID-19
• Competitive Playback Technology Enabling Live Commerce to be More Perfect
• Ultra Low Latency, Time Machine, and Optimized Thumbnail-Seeking
• Global and Mass Traffic Caused by the 2021 Tokyo Olympics
• Live Playback Technology for the Future

Targets

• Developers and planners, who work in the mobile live streaming field
• Developers, who are interested in live publishing and playback
• Developers, who are interested in media and network technology for stable publishing and playback

Need for Integrated NAVER DB - Beginning of Stella KB

NAVER DB Integration process

• Ontology
• convert to RDF triple
• DB Linkage / Entity Alignment / Entity Resolution
• link to Open Data

Ontology

• Taxonomy design
• property / predicate / relation design

DB Linkage / Entity Alignment / Entity Resolution

• For Unstructured DB
• For Structured DB

Stella KB Applications

• KB QA
• Information recommendation
• Link NAVER Services

In this session, we will share the experience of deploying a service that we struggled with in the keyword recommendation project to recommend various keywords to users in a situation where logs are scarce.
In order to automate the hot topic keyword recommendation manually operated by editors, a real-time topic detection model and a method to extract search keywords from the hot topic issues by using HyperCLOVA are introduced.
We also would like to share an example of expanding the keyword pool using pattern mining methods to recommend various keywords that fit the user's preference.
The overall service pipeline of the Line Search keyword recommendation project and keyword recommendation methods will be introduced.

Contents

1. Overview of Keyword Recommendation Project

• Overview of Line Keyword Recommendation
• 2020 AiRS AB test
• Emerging Issues
• 2021 AiRS TODO
• Overview of Model for Keyword Generation

2. Automated Generation of Hot-issue Keywords by Leveraging HyperCLOVA

• Let's automate the manual keyword generation operated by human
• Real-time Hot-Issue Detection Model
  • Latest Popular Model
  • Cluster-based Model
  • Future Impact Model
• Model of Keyword Generation
• Few-shot Learning
• How to improve the quality of automated keyword generation?
• FEw-shot Example Tuning
• Fine-tuning
• Evaluation Result
• Limiation of Automated Keyword Generation

3. Keyword Expansion and Personalized Recommendation for Diversifying Search Experience

• Overview and Ideas
• Sequential Pattern Mining
• Eliminating Inappropriate Words
• Result of Observed Patterns
• Quality Improvement
• Keyword Embedding
• Flow of Keyword Recommendation
• Example of Keyword Recommendation

4. Ranking & System Pipeline

• Lessons from 2020
• Pipeline of Rank1 Keyword Selection
• Pipeline of Keyword Recommendation

5. Future Works

1. Motivation:

• Motivation to study vision and language reprsentations with different task in a shared model.

2. Introduce ViLBERT which learns task-agonostic visionlisgit features through pretraining.

• Transfer learning from pretrained network is a very popular scheme in both vision and language comunities.
• Describe the model, dataset and pre-training process.
• Describe how to transfer to different downstream tasks.

3. Introduce ViLBERT Multi-Task, which is a single model trained on 12 tasks.

• Motivation behind a single model for different vision and language tasks.
• Descibe how to train a single model on 12 vision and language tasks.
• Point out problem is many datasets carve up the same image sources differently.
• Descibe dynamic stop and go training schedules.

4. Intdouce X-LXMERT, which is a multi-task model for image generation tasks.

• Motivation behind the image synthesis tasks for vision and language tansformers.
• Describe LXMERT and prob LXMERT to paint.
• Propose X-LXMERT model, which is LXMERT with a display server.

Today, the development of big data and machine learning technology and the establishment of a legal system to utilize personal information, various services using personal information are being released. But, the risk of personal information leakage is also increasing. De-identification technologies can be used to use personal information, but existing de-identification methods have limitations. In this lecture, we will use the homomorphic encryption, known as the next generation cryptography technology, to protect personal privacy and introduce HEaaN Homophobic Analysis, a recently launched the homomorphic encryption service by Naver Cloud. Finally, we provide insights to devise various services using the homomorphic encryption by looking at overseas and domestic cases using the homomorphic encryption.