Huawei experts discuss how to become an excellent engineer!

Thread Content

Whether at companies T and W or at Huawei, I have been fortunate enough to work in the OS (operating system) industry. But to be honest, it is an industry full of endless challenges as well as pleasures; it is often not as glamorous as people think. When many people ask me what I think of operating systems, I always say this: An operating system might be the most important thing in the world, but it is also the most useless. The most useless aspect is that an operating system only gains value when it is integrated with chips, business applications, and so on ； It is most useful because the operating system serves as a bridge between chips, hardware, and business operations; whoever controls the operating system holds the upper hand. My career has always been inseparably linked to operating systems. The openEuler community, initiated by Huawei and representing a truly open community in the field of operating systems in China, seems to be bringing new changes to my destiny. Even if it is the will of fate, it does not prevent one from drawing some insights from one’s career. As an engineer, I think there are generally four stages to go through. 01 Sustained interest: Interest is the foundation of success. Although I attribute my career to the workings of fate, in reality, if I hadn’t had an interest in this industry, I believe my life path would have led me away from it countless times. That’s why I’ve always believed that interest is the only thing that can help us go far in our professional careers. I first became involved with Linux and operating systems over 20 years ago, when I was in college. During my junior year, a classmate returned to school from Beijing, bringing two items: an installation disc for Company T, and a manual on embedded OS development from Company W. After all these years, I can still clearly remember the red and white lightning pattern on the disc, as well as the nearly completely black cover of the development manual. I was holding those two \"treasures\" at the time and thought: If I could work for a company like this one someday, it would be the pinnacle of a life. But at that time, I could never have imagined that, years later, I would become the R&D leader for these two companies. What I learned during my master’s and doctoral studies had nothing to do with OS; my field of study was distributed computing and large-scale storage. Just as my life seemed to be moving smoothly along its predetermined path, graduation came around and I found myself in the awkward situation of not having a job. At that time, China’s IT capabilities were still at such a level that simply acting as an agent for Western products was enough to make money; few companies invested in in-depth research and development. Fortunately, the need to modify the Linux kernel for storage purposes, combined with my interest in playing around with Linux, helped me. Although my skills at that time were quite rudimentary by today’s standards, in an era when being able to install Linux already made one a skilled person, those skills were enough for me to pass T Company’s interview and join the company I had long admired since my university days. The working conditions at that time meant having to do everything, and having to rely on oneself for everything. I quickly realized that it was nothing like the glamorous experience I had imagined back in school; developing Linux distributions is actually a very dull and tedious task. There are no hackers with big beards, nor is there any open-source style of writing code in T-shirts; instead, there’s an endless stream of software packages to create, installation programs full of countless bugs, and program crashes and system freezes that can occur at any moment. I was awakened at 2 a.m. by sales calls, and it was common to have to answer technical questions; this inspired me to write a paper titled: \"A Sociological Analysis and Technical Background Study of Why IT Failures Always Occur in the Late Night Hours.\" As the market and industry evolved, after working at Company T for 4 years, I joined Company W’s WRLinux product line. Since the product line was newly established, it went through a chaotic period; we had to work together with colleagues around the world to gradually get the product line on track, handling everything from product design and implementation to process management. After working in companies for many years, I realized that no matter what industry it is, boredom is ultimately the essence of that industry. No matter how much I admired and looked forward to companies T and W back when I was in school, and no matter how great my respect for OS technology was, my initial interest faded not long after I started working, replaced by fatigue and boredom. This is actually quite normal; once any interest turns into a career, what remains is mostly just pain. It is perhaps at this point that interest determines whether you truly enjoy doing this work, or whether you are capable of doing it; in fact, interest plays a significant role in determining how far you can go in this industry. I find it hard to imagine someone who has no interest in operating systems turning on their computer at home after work to try learning a new language, to install a new OS to experiment with, to understand how the underlying mechanisms of the kernel work, or to write a small program out of boredom just to keep practicing. I still prefer to use a quote from Linus at the CNCF 2017 conference to illustrate the importance of interest. He said he still hated C++, and still felt a mix of affection and resentment toward the maintainers, but when asked how to run a software project well or how to make an open-source software more successful. Linus said, “When you’re creating software, don’t worry about whether it will be successful; you just need to consider whether you enjoy doing it, and then go ahead and do it.” ” 02 Technical foundation based on details. Admittedly, just having interest is not enough; one needs the necessary capabilities to achieve anything. To gain an in-depth understanding of a particular field, it is essential to have a solid foundation. A solid foundation means climbing up step by step from the lowest levels of this industry, working through all its various processes and gaining hands-on experience of the entire workflow. Only in this way can one gain an understanding of all the technical details, and it is the details that always serve as a measure of whether someone is professional. On this point, I would like to thank my past experiences; I went through the entire process of creating a Linux distribution, carrying out all the necessary steps. No matter where there is a problem, it needs to be repaired, and this has also helped me gain a comprehensive understanding of a massive system like OS. The conditioned reflexes formed by these experiences remain clear to this day. This is the foundation of technical depth. Another aspect is the foundation of technical breadth; although my research focus during my school years had nothing to do with operating systems, many of the basic theories and design methods in distributed systems and storage systems have still been of great help to my career path later on. The mental training during my doctoral studies, along with the emphasis on details, also had a profound impact on me. At one point, I suddenly realized that, whether it’s large-scale data centers or the design of individual processors, the entire IT world revolves around the fundamental principle of \"consistency\". It is completely impossible for these chemical reactions to occur without a foundation of basic knowledge regarding the relevant details. At the same time, the professional habits developed through long-term attention to details greatly contribute to technical judgment. In the OS field, problems are usually rare, but when they do occur, they are serious ones. To give an example from my own experience, a few years ago a customer’s USB system encountered problems. The customer and the FAE (Field Application Engineer) team conducted a thorough analysis and determined that the version range in which the issue occurred was from XX to XX. However, the problem had to be resolved within 2 days, otherwise the consequences would be severe. It seems not that difficult; by following the binary search method, it’s possible to quickly identify which patch was integrated and caused the problem. But I keep feeling that something isn’t right ； One issue is that although this incorporated code is related to USB, the content of the patch does not exactly match the problem ； Secondly, having been in this industry for a long time, I have developed a suspicious habit; although the previous teams assured me repeatedly, I have always doubted the conclusion that problems always arise between XX and XX. In the end, in addition to continuing to investigate the cause using binary search, I enlisted another engineer to go over the issue in detail, and I also conducted a thorough comparison with the entire community, without limiting myself to any specific version range. Fortunately, the engineers eventually found an unremarkable patch that had nothing to do with that version range and resolved the issue; my caution and carefulness also helped the company retain that client. No matter how senior an engineer is, only by paying attention to details can they avoid certain errors that might seem obvious at first glance. Moreover, having a broad range of knowledge enables one to possess sufficient analytical and judgmental skills, to have a good understanding of the field in which they work, and to facilitate interactions between different areas of knowledge. 03 Deep understanding of the customer’s industry: Building on interest and technical details, an even more important step is to have a thorough understanding of the industry in which the customer operates. From the very beginning of my career, I had the opportunity to be in constant contact with the market, and I was involved throughout the entire process of developing commercial operating systems for various chips. I was involved in technical analysis and demand negotiations, and I was even the one who came up with the initial business quotes. Although this may not seem to be directly related to pure technology, it is incredibly meaningful for an engineer to be able to see firsthand how what they have created turns into a product. Wearing a suit and tie to meet clients or bid on projects does not fall under the category of technical skills in the traditional sense, but I have always believed that such experiences are far more important than my technical expertise. It cultivates a certain kind of “intuition,” the industry’s “intuition.” Although I think that, for many things, human intuition is unreliable, that applies to matters that are certain. However, many things in the world are uncertain, and there is no other way than “intuition” to judge the direction of such uncertain things. Participating in the processes of marketization and product development can greatly enhance engineers’ deeper understanding of technology. Taking openEuler as an example, from the very beginning of the design process, we have always emphasized that the top priority in design should be the interests of our ecosystem partners, with the goal of helping them achieve greater profits. From a purely technical perspective, it seems strange – we incur costs without actively seeking to generate any benefits. However, from a broader market perspective, the operating system itself serves as a bridge between hardware and software; when this connection is smooth, communication becomes more efficient and the market thrives. The players in this market, whether they are in the hardware or software sector, can all achieve substantial profits. Therefore, there is absolutely no need to charge even a small toll! Only on the basis of a deep understanding of the industry can we discuss technology planning, the path of version evolution, the life cycle, and even topics such as community governance and investment in technical directions. If one were to summarize it, it can be expressed in one sentence: not only must one understand technology, but also the industry itself. 04 Innovation for Complete Business Solutions: I have always believed that innovation is only possible when one has a thorough understanding of the field in question. By innovation, I do not mean merely an improvement in a technique or a quick optimization of some code; rather, it refers to changes of varying scale that stem from a deep understanding of the industry, whether in the form of a complete product or a complete system. Moreover, such changes, products, and systems should be able to generate commercial benefits. This should be the highest level a engineer can attain. From an innovation perspective, in my opinion, a top-notch engineer is not someone who draws a line on the engine’s casing and claims that line is worth $1, but rather a genius who knows where to draw the line so that it’s worth $49,999 – it’s the anonymous engineer who actually creates the engine. In my opinion, that genius merely fixed a bug in the engineer’s work; it was just a ‘minor innovation’, while it was the engineer who gave this story its beginning. Traditionally, internally driven innovation led by companies emphasizes the need for a strong integration with business operations, as well as the requirement for immediate returns. As a result, the R&D environment within companies may not be very conducive to many innovations that require a long time to develop and could have disruptive effects. Fortunately, Huawei has officially begun to implement its Kunpeng industry strategy, with the operating system playing a key role in this. The company established the openEuler open-source community, building an ecosystem in line with industry practices. Its goal is to serve as an open and innovative open-source platform that provides a solid foundation for third-party operating systems. A brand-new Kunpeng architecture, and a brand-new open community for openEuler – we can bring in some immature ideas, or even those that seem highly speculative, and nurture them within this community, allowing the tolerant open-source community to provide a favorable environment for such projects to grow. I hope the openEuler community will give rise to more engineers who create engines, and the only difference is that you will no longer remain unknown. 05 Conclusion: The company has undergone many changes in recent years, and for me, the changes in culture and mindset are the most significant. And terms like trustworthy, ecological, and clean code – though they may seem somewhat metaphysical – are now gradually causing certain positive effects in various parts of the company, in different forms. However, I deliberately refrained from discussing any experiences and insights related to coding. I’ve always believed that although coding skills are important, they can generally help an engineer reach only the second level—the basic level. Just as from kindergarten and elementary school we start practicing writing Chinese characters and learning writing skills, knowing Chinese characters and having those skills does not guarantee that we will be able to create great works. Any great literary work is, at its core, a profound understanding of life. Therefore, in my view, apart from climbing through these four levels step by step, there seems to be no other way for us to reach the top instantly and lead the world. Therefore, when our various departments and teams pursue innovation, it may be necessary to first examine whether they have gone through these stages and whether there are enough people who meet the requirements at these levels. Here are some personal suggestions for young engineers. Over the past couple of years, the industrial landscape has undergone tremendous changes; on one hand, hardware areas such as instruction sets, chips, architecture, and storage devices have seen significant transformations ； On the other hand, emerging fields such as 5G, cloud computing, IoT, and edge computing are experiencing rapid growth, all of which are driving profound changes in the field of operating systems. These systems are not only bridges connecting hardware and software but have also become the controllers of software ecosystems. Operating systems and the ecosystems that surround them have become a battleground for major companies, bringing this traditional field back to the center of attention in a new form. I hope that more and more young people will develop a strong interest in operating systems, build a solid technical foundation, engage with the market, gradually accumulate industry knowledge, and unleash their creativity. I also hope that companies can create such an environment for engineers, especially young ones.