There are many nice articles about web site performance optimization accross the Internet. As long as this is really important topic, I think there can't be too many of them. That's why I'd like to share some suggestions on how to make Kentico run faster.
Generic
Scheduled Tasks: turn off any unnecessary scheduled tasks; make sure all enabled tasks are running with meaningful frequency
Output Filters: output filters goes through rendered page markup and fix some elements within it. If you feel confident about quality of your HTML you are good to turn them off. Keep in mind there might be a lot of markup added by content editors through editable text web part
Caching: Kentico provides us with great caching capabilities out of the box - do not hesitate to use it! Data caching is a key to a good performance. Kentico allows us to cache almost everything and almost everywhere. With cache dependencies we can easily specify when system should update cached data. We can cache data retrieved with data source web parts, viewers, macro, page data and any data we read from database in custom code. It is extremely useful with custom transformation methods
Macro: make sure all macros are signed with valid signature
Event Log: check event log for any errors in it and resolve if any
Debug: make sure debug is disabled in production. It could be enabled only in case you need to troubleshoot some issue and should be disabled immediately after completion
Continuous Integration: CI is developer’s tool to synchronize their working environments. It should never be used to transfer changes from QA to production, which means it should never be enabled in production
CSS: put all you styles into (one) Kentico stylesheet. Try to avoid adding CSS into transformation, web part containers, etc.
JavaScript: do not place too many web parts with inline javascript. You should better place your code in js file and reference it, so you may substitute many inline javascripts with one file
Resources compression: allow resources compression and minification of JS and CSS. This will increase processing for the first request, but speed all furthers ones
CMS Tree: keep your CMS Tree organized and well structured, avoid placing 100500 pages under the same parent, use a couple of levels instead. In addition to performance benefits you get, this will allow content editors to navigate easier
Template
Web part IDs: do not populate/use web part name, you can always specify some meaningful ID for web part, which will help you to identify the responsibilities of the actual web part
Content before/after: Kentico provides us with web part containers, which is very convenient way to wrap web part with some markup. Unfortunately, is not efficient, as it adds extra loading and processing. Content before/after is much more efficient, as it loads along with other properties of the web part, however it is not convenient as it requires knowledge of HTML
Check permissions: make sure this property of the web part is set to false in case of public content. This also could be setup on the global level in Settings in case of publicly available web site
Columns: make sure you're loading only data you need. If you leave this property of a web part blank, Kentico will load all available column. In some cases, number of column could be around 200. Usually we need 5-20 columns on page
Data storage
Files: store files in file system - it is much faster to load them from there. Also you do not overwhelm your database with gigabytes of files
Media: store all media files into Media Library instead of CMS Tree
Data: often we choose custom page type as best fit to store some data and/or content. Unfortunately, page types are not efficient performance wise. Kentico stores pages’ data into three different database tables and has a lot of processing around it. There are many benefits and features available for page types, those are not available for other objects: workflow, security, localization, same page type stored it in different locations, etc. But there are situations, when we do not need any of those features. In those cases, I'd recommend you to consider Custom Tables or Custom Modules. With any of those two you can easily provide content editors with nice interface to manage those objects. This also help you to prevent enormous amount of pages.
Conclusion
There are many other things you could do in order to increase performance of your application: placing javascript files references at the end of your page, using CDN, optimizing of your images, combining javascript and css into a single file, loading data with AJAx, emilimination of the errors in the markup and javascript, etc., but all they are out of Kentico topic, however also very important.
Tips I've mentioned here are not requirement and might not be even possible in some cases, but be sure to consider them.