Data Senior Engineer

  • Comptitive
  • Beijing, Beijing Shi, China
  • 1-3 yrs
  • Starquest Capital Investment Management (Shenzhen) Co. Ltd
  • 31 Oct 18

Data Senior Engineer for PE fund based in Beijing,China

Responsibilities:

负责多平台信息挖掘和页面内容的提取分析;
参与系统的架构设计和研发,独立进行抓取和清洗工作;
负责技术难点的攻克;
负责大规模数据挖掘的性能优化工作;

 

Qualification:

计算机相关专业,本科及以上学历;1-3年以上实际数据开发、内容提取工作经验;能独立解决实际开发过程碰到的各类问题;
熟悉Java 、Python等语言的开发,熟悉Linux命令;
深入了解Http协议和web登录认证机制,熟悉web前端,可读懂js代码;
了解多线程、多进程、网络通信编程相关知识,有过海量数据挖掘经验、清洗项目经验;
掌握网页抓取原理及技术,了解基于Session的登录原理,熟悉基于正则表达式、XPath、CSS等网页信息抽取技术;
了解APP模拟及接口验签破解技术,了解APP用户授权访问机制,掌握Selenium,APP破解技术优先;
工作认真细致踏实,优秀的学习能力与抗压能力。

--------------------------------------------------------------------------------------------

Responsibilities:

Responsible for multi-platform data mining and page content extraction analysis;

Participate in the system design and development, and independently carry out the crawling and ETL work;

Responsible for technical difficulties;

Responsible for performance optimization of large-scale data mining;

Qualification:

Computer science related, bachelor degree or above; 

1-3 years of experience in actual data mining development and so on; 

Able to independently solve various problems encountered in the actual development process;

Familiar with the development of languages such as Java and Python, familiar with Linux system;

Deep understanding of the Http protocol and web login authentication mechanism, familiar with the web front end, and understand the JS code;

Understand multi-threaded, multi-process, network communication programming related knowledge, experience in massive data mining, cleaning project experience;

Master the principle and technology of web crawling, understand the login principle based on Session, and be familiar with web page information extraction techniques based on regular expressions, XPath, CSS, etc.

Understand the APP simulation and interface technology, understand the APP user authorization access mechanism, master Selenium, APP crack technology is preferred;

Work carefully and practically, with excellent learning ability and ability to withstand stress.