Didi Caole: How to become a technology master?

All tragedy erased, I see only wonders…

Didi Caole: How to become a technology master?

On November 12, 2022

> Jumei Guide: Cao Le, graduated from Tsinghua University, joined Didi in early 16 years and led the team to build Didi online ride-hailing technology system, He is currently the head of Didi’s ride-hailing technology department. Faced with the confusion of the growth of the technical team students, Cao Le wrote a letter to the students, he clarified his insights and ideas from various dimensions, helping the students to no longer limit themselves to looking at problems from a technical perspective, but to have a broader vision and methods. He put forward the following ideas around the topic of how to become a technical powerhouse: finding paradigms, deliberate practice, timely feedback; Vertical penetration, lateral migration, deep review; Smart people have to work hard. I would like to share the contents of this letter with you again.

Many students have questions about how engineers should grow, and everyone is generally confused about how to grow into a cattle person, how to get promoted, and how to continue to learn in busy work, which is actually the only way for every student to grow up. I would like to share with you some of my experiences.

Students are generally full of anxiety about growth. Too busy at work and no time to study, too many needs and too trivial to feel that I have not made much progress, whether to do technology is to achieve that no one wants it after the age of 35, and so on, are all manifestations of growth anxiety. This anxiety is normal, and all desires are projected in the heart is actually anxiety. Anyone who aspires to grow, no matter what stage they are in, front-line engineers, architects, directors, and vice presidents, is actually full of anxiety in their hearts, without exception. What we need to do about this anxiety is to accept it without worrying too much. This anxiety does not mean that if you figure out how to grow, you will be gone, and at some stage it will be gone. The pace of growth and expectations do not stop for a moment, and the anxiety in my heart will not stop for a moment. It is this sense of anxiety that drives you to write code to track down problems to the starry night, drives you to sacrifice time for rest and entertainment and the company of a thick and boring book, and drives you to constantly strive forward day and night. On the contrary, if there is no such anxiety in the heart, it is worth worrying about. This may indicate that you are used to staying in your comfort zone. In such a fast-growing society and our fast-growing and changing industry, losing the desire and anxiety for growth is a very dangerous signal.

The so-called programmer 35-year-old crisis, in fact, the fundamental reason behind it is that too many people feel that they can do everything after working for a few years, and the next ten years of work are just a simple repetition of the first 2-3 years. In an industry like ours, when recruiting, if there are two people in front of management, one is a fledgling or has just worked for 2-3 years, full of desire for growth; The other person who has been working for more than ten years but at about the same level as the person who has worked for 2-3 years, only more skilled, but has been lying in the comfort zone for ten years. If you were the one in charge of recruiting, what choices would you make?

On the other hand, in fact, high-end talents are extremely scarce in the industry, which is a very common phenomenon in the industry, and the real big cattle are too scarce. In such an industry, if a person can continue to grow, ability and working years are directly proportional to continuous improvement, such a person, at any time in the industry is crazy, how can he encounter any age crisis?

How to learn actually has a methodology, that is, deliberate practice. The so-called theory of 10,000 hours to become a big cow is one-sided, and it is impossible to become a big cow if you simply repeat 10,000 hours. Deliberate practice consists of three steps. First, find the pattern of the domain system you want to study; Second, deliberate repetition and practice for each paradigm; Third, timely feedback.

In the past work and study life, everyone has been practicing more or less deliberately. Take middle school students facing the college entrance examination as an example, good students usually break down a course into a lot of knowledge points (looking for patterns), and then for the knowledge points and their permutation combinations, targeted and repeated questions of various difficulties (deliberate practice), each time after completing the question, check the answer to see if it is correct or not, if it is wrong, think, record, review (continuous timely feedback). This way of learning is to do more with less. And the learning method of half the effort, is to indiscriminately pick up an exercise or paper and do it desperately, when I was in school, many classmates around me were very diligent but their grades were not good, most of them were for this reason. To take another example of me recently learning to play badminton, the correct way to learn is to disassemble badminton into footwork and hand movements, small broken steps, meter steps, forehand and backhand picking, putting the net, forehand and overhead high ball hanging ball to kill the ball, etc. (looking for patterns), and then practice repeatedly for each movement (deliberate practice), and then ask the coach or record to watch the video to correct their movements (timely feedback); The wrong way to learn is to blindly find someone to play the game and practice with the race, such progress is very slow, and the wrong action is difficult to correct in the future after forming a habit.

When the study method is incorrect, hard study often just seems diligent and has no desired effect. When exposed to an unfamiliar field, the wrong way to learn is without purpose, and come up and find a bunch of relevant tomes to start nibbling. And the correct learning method should be to quickly sort out the knowledge points in the field, form a framework system (looking for patterns), here are some tips to quickly build a field of knowledge point system, such as reading some review or groundbreaking articles in the field (read papers, don’t blindly read online articles), or find a review textbook in the field to see its table of contents (note that the catalog of good textbooks is often the knowledge framework of this field, and the content does not necessarily have to be read). Then, for each knowledge point, find the relevant chapters in the book, and the relevant sections in the relevant paper in the field to learn deeply to establish your own understanding of this knowledge point (deliberate practice). Finally, the knowledge points are compared with the situation in real work (their own work, or other company-related work) (timely feedback) to establish a deep understanding of a knowledge point, and finally integrate to establish an understanding of a field.

This may be a bit abstract, take the process of distributed storage I learned as an example, first combine my own work content to sort out the knowledge points that need to be in-depth (for example, meta-information organization, Meta Server design and HA, copy organization and management, Recovery, Rebalance, stand-alone storage engine, data/meta-information flow, erasure coding, consistency, multi-tenancy, storage media, network environment and IDC, etc.), while reading a lot of summary materials. Sort out the knowledge points of distributed storage (there are various better articles organized on the Internet, and there are also abstracted from the papers implemented by various systems), and constantly iteratively build knowledge points in the field of distributed storage (finding patterns, which is the most difficult process); Then for each knowledge point, find relevant materials for deep learning, for example, for distributed consistency, you need to read CAP theory, Paxos’ papers, Raft’s papers, etc., and a lot of materials around (deliberate practice); Then find papers or articles implemented by various systems, such as GFS, Dynamo, Aurora, OceanBase, Ceph, Spanner, etc., to see and compare how they are considered and trade-offs in consistency, of course, the most important thing is to combine the repeated practice in their own work and the knowledge learned to compare (timely feedback).

These three stages are not separated, but repeated, often in the process of deliberate practice and timely feedback, they find that they have missed knowledge points, or find that the two knowledge points they sort out actually overlap. The knowledge points established through this cross-comparison and continuous testing in practice are very landable, and will not be clouded after reading a few papers. Take the consistency of distributed storage as an example, if it is not repeated comparison, thinking and repeated practice, you will not find the most difficult paragraph in the GFS paper, the logic of multiple writers appending a file, which is useless in practice; You will also not find that the seemingly elegant and academic CAP theory of choosing two out of three, in practice is not so perfect at all, many times you can only choose one of the three; You won’t find the Vector Clock in Dynamo’s paper, there are countless articles on the Internet shaking their heads to interpret, but in the Amazon application scenario is a typical over design, Cassandra is much more pragmatic in this regard.

At this time, you

may have a question, the work itself is so busy, where can you find enough time to study?

In fact, work and study themselves should not be separated. Work should be part of learning, part of practice and timely feedback in learning. Learning is actually very inefficient if it is divorced from the practice of work. Therefore, each student should systematically learn the technical and business field in which he works, and repeatedly practice and verify it in his work. Different fields are actually integrated, and when you are proficient in one field and summarize the methodology, it is easy to get started with other fields. Therefore, spending several years of practice to thoroughly study a field is very important, even necessary, for students who have just worked for a few years, and only after a field is thoroughly understood can they talk about cross-field migration to expand their knowledge. To put it more directly, for students who have not fully mastered a field, depth is the most important, do not think about breadth, and after mastering a field, it becomes easy to expand the breadth.

A common misconception here is that there is not much direct relationship between what you learn and the field of work. For example, I used to spend a lot of effort to read the Linux kernel source code and many related tomes, spending almost all my free time for nearly two years, but in my work over the years, it was almost useless, at most there were some “inspirations”, the ROI was so low that I have almost forgotten about it now. More importantly, software engineering is a practical science, and the knowledge obtained from books is basically useless if it is not applied and tested in practice. For example, many excellent architects, although they may be used repeatedly in their daily work, may not be able to say the opening and closing principle, the Richter substitution principle, the Demeter Law, etc., in turn, many people who export the 7 principles of object-oriented design are actually far from real architects, and some are even just blog architects. Practice is far more important than reading books and articles, and the process of building my own distributed storage knowledge system described above seems to be reading materials and papers, but in fact, 80% of the gains come from practice with theory, and summarize the precipitated theory from practice. Therefore, it is the most pragmatic and efficient way to thoroughly understand the technical and business areas in which you work, and the separation of work and study will lead to work and study not doing a good job.

At this time, you may have another question, feel that the daily work is very trivial, can’t learn anything, what to do?

If learning is divided into learning from books and learning from work, then there is no doubt that the “knowledge density” at work must be much lower than the “knowledge density” in books, because the knowledge in books is abstracted from their work. This is why everyone generally feels that daily work is “trivial”. However, every little bit of triviality and ordinariness in work can be abstracted into a methodology, not to mention the breadth and depth of the field in which the work is located. The secret to learning from daily work is to “think in action”.

For every software engineer, the two most important abilities are the ability to write code and the ability to trouble shooting. And, to be a good architect, great development skills and the ability to trace problems are the foundation of everything. The core of improving the ability to write code is, first of all, to insist on continuous writing, but more importantly, to continuously review your previous code every day, weekly; Also, review more code written by people, such as colleagues on the team who you think write better code than you, such as open source code in the community that is known for its beautiful code (as a C++ programmer, one of my role models back then was the boost library). Once you feel that your previous code is not good enough, immediately review and refactor immediately. More importantly, think more about the reasons behind the differences between your own code and good code, and often that’s the secret behind the better code.

In particular, in addition to knowing what a code specification is, it is important to think about the reason behind each code specification. Behind every sentence of the code specification is a history of blood and tears on the rivers and lakes. To improve the ability of trouble shooting, the key is to deeply review every problem you encounter, including online, including what is found in testing, looking for every problem, the root cause behind each accident, and thinking about how to avoid similar problems in the future, how to find similar problems faster. Be curious about all problems encountered inside and outside the team, and pay attention to the surrounding accidents and root causes behind the problems. The improvement of trouble shooting ability is almost impossible to get from books, and comes from deep thinking about each problem and extensive accumulation of each problem. For architects, it may not be possible to write code on the front line, but a very important criterion to see whether an architect on the team is really good is to see whether he can track down problems that other students in the team can’t find. A real cow architect I have met, for difficult diseases in the system, usually ask a few questions, you can roughly guess where the problem is and what the possible causes are, the accuracy is like fortune telling, tried and tried, breathtaking.

For an architect, in addition to better code ability and trouble shooting ability, it is necessary to build a relatively complete knowledge system in the current technical field, need to have systematic thinking ability, and need to have a very deep understanding of the business served by the technology. Systematic thinking ability comes from two aspects. On the one hand, in daily work, every interface design, every logic, every module, subsystem split and organization, every required technical solution, and every top-level design of the system must be repeatedly thought and deliberated, and constantly reviewed. On the other hand, it is necessary to learn a lot of architectural design of similar systems in the industry, which is actually an eye opener, but relatively speaking, the communication in the industry is more frequent. Taobao, Meituan, Baidu, Google, Facebook, Amazon and other companies introduce the system architecture papers and PPTs, and need to continue to learn with questions. In addition to the technical field itself, architects need to have a good understanding of how our systems are used in business, otherwise it is very easy to over-design and fall into the self-concession of technology, which is why I say that the Vector Clock in the Amazon Dynamo paper is an over design.

On the other hand, many times the technical hurdles that cannot be bypassed, may be very complex implementations, often only need a little change in the upper business, can be completely bypassed, which is why I said that in the GFS paper, multiple writers and Append the same file is a useless design at all (in fact, Google has also removed this function internally). Only when you really know how the upper-level business uses the system can you really do a good job of architecture. It is not difficult to understand the business in depth, for each student, as long as for each received demand, for each demand in the review of each demand, for the surrounding students or team to do the demand, think deeply about why the business should put forward this demand, what problem this demand solves the business, and whether there is a better solution. If you encounter something you don’t understand, ask more students around you, products, and operation students. The most afraid is that I limit myself to pure research and development, and I have no brain to do it when I receive a demand, which is equivalent to giving up active thinking. There is also a way to measure whether a person is a good architect. For a requirement, if he gives several feasible solutions, saying that these solutions are okay and those solutions are okay, it often means that he has not fully entered the architect’s path. The difficulty of the architect is not to come up with a solution, but to find the only one that is the simplest and most elegant solution.

To sum up, thinking in action is to always be curious, constantly find problems from work, and constantly return to work with problems; Constantly think, constantly verify and think in work; Constantly summarize abstractions from work, constantly review the work, continue to cross-verify the work content and knowledge in the whole field, and repeatedly practice the process.

Deliberate practice in the technical and business fields where you work, coupled with thinking in action, is the secret to becoming a technical powerhouse.

It seems that the method is not complicated, why are large cattle still very rare?

Although we talk about methods throughout, in fact, on the road to becoming a technology giant, methods are not so important. The real difficulty lies in years and decades of perseverance. Too many people encounter setbacks and bottlenecks, they feel that the things at hand are too boring and boring, and they want to change directions, change fields, and learn new technologies and new things. And those who can really become big bulls must be able to green the ancient Buddha, survive the loneliness for a long time before breaking through the bottleneck, and must be smart people who are willing to make stupid efforts. Therefore, the method is actually not so important compared to persistence.

Author of this article

▬