• Flink The community itself iterates very fast, at present, Alibaba Cloud has a large wave of people full-time to do Flink open source, in addition to having active community contributors, so the function development is faster, the bug repair speed is faster, almost every 4 months a major version, between each major version iteration between the functions is very many, the code changes are very large, the API interface changes are also large, and they can easily overturn themselves.

• Community iteration is fast, why the company Also to constantly follow the community’s nose? Fast community iteration means more features, more bugs fixed, and higher stability than earlier versions. In addition to the domestic first- and second-tier companies have a lot of full-time people to be responsible for this piece, most small and medium-sized companies have the simplest and fastest experience of the most stable, most functional, and best performance Flink version is nothing more than directly using the latest Flink version. For example: Flink SQL from the earliest (1.9) function and performance to the current 1.14, the difference is really much bigger, optimizing a lot of places, enhancing a lot of functions. Originally, using Flink SQL to complete a stream processing task was very troublesome, and it was not as fast as writing dozens of lines of code directly, but now I prefer to write SQL to process a stream task. Then it will naturally follow to upgrade to the new version.

• User A Q Does Flink SQL support setting parallelism separately? User B Q Does the live platform now support Flink version 1.13 of Window TVF? This can only be supported by the Flink xxx version, or do you upgrade the Flink version to xxx? In this way, it can be supported, and there are many similar scenarios, which is nothing more than the most trouble-free for the head of the real-time platform of small and medium-sized companies; For those responsible for real-time development in large companies, this is undoubtedly a nightmare, every time you upgrade a new version, you have to try your best to port the various functions developed in the old version to the new version, and encountering large changes in the API interface is nothing more than rewriting, or some of the special needs of the new version are patched into the old version.

• The new version of incense is really fragrant, but why don’t some people use it? The problem is that most real-time jobs are long-running, if a job has no errors, runs well in production, does not have any failures, stability and performance are acceptable (not all jobs have a large amount of data, will encounter performance problems), then users Why use the new version? Users don’t care how awesome your new version is, how good the performance is, Laozi upgrade also needs to change the dependent version, change the interface code, test joint debugging, performance testing (who knows if the performance improvement you said is bragging), stability testing (may be online double run for a period of time verification), these do not need time, you tell me to upgrade and upgrade, roll the calf, do you know how much business needs I still have to do?

Then it falls to this venue, and to use the new version of the function to solve the problem, the user of the old work and his various disputes can not impress him to upgrade the version of the operation, then naturally there are multiple versions.

In this way, if you don’t plan for the version, then the stall will gradually get bigger and bigger, and it will become more and more difficult to clean up?

So how do you manage your company’s version of Flink? How do you manage and submit jobs that are compatible with multiple Flink versions? How to be compatible with JAR packages and SQL

job

submissionsHow to manage job submissions for multiple Flink versions?

Stay tuned for the next article