The concept of 'heat exchanger efficiency' is not very well defined in technical literature. Efficiency of any system has to be calculated based on the performance of some ideal system. In case of heat exchangers, we don't have very well defined 'ideal' heat exchangers.
Overall heat transfer equation
For any heat exchanger system, the overall heat transfer rate (Q) is defined as -
Q = U×A×ΔT
where, U is the overall heat transfer coefficient
A is the overall heat transfer surface area
and ΔT is the mean temperature difference between hot and cold side
Ideal heat exchanger system
In most cases the mean temperature difference is a given. The heat transfer rate is mostly governed by the product UA, which is the product of surface area and heat transfer coefficient.
UA can always be improved by changing the heat exchanger design or just increasing the surface area by getting a bigger heat exchanger. For that reason, we don't have a practical definition for 'ideal heat exchanger'.
Calculating heat exchanger efficiency
For any system, efficiency is normally calculated by comparing the actual performance with ideal performance.
Efficiency = Actual out put / Output of the ideal system
Since we don't have an ideal heat exchanger to compare with, we cannot use the traditional concept of efficiency for heat exchangers.
Instead we can use the value of 'UA' (product of surface area and heat transfer coefficient) to represent the best heat transfer rate that can be achieved for given process conditions (ΔT).
When you are comparing between different heat exchanger designs, it is a good idea to compare their UA values. That should give you a good idea of how much heat transfer would be possible with each of them.
To further examine the economic viability, you can also compare the ratio of (UA/cost). That will help you to identify a heat exchanger design that can give you the best heat transfer rate at the lowest cost. This approach will give you the most productive heat exchanger design.
Heat exchanger effectiveness
When a heat exchanger has been in operation for a while, its performance decreases for a variety of reasons - such as, fouling, scaling, corrosion etc.
This reduced performance can also be quantified as the reduced effectiveness of heat transfer. You can compare the current heat transfer rate to the original (rated) heat transfer rate.
A ratio of (Q/rated duty) will tell you how effectively your exchanger is working.
Shell & tube heat exchanger efficiency
For shell & tube heat exchangers, the overall heat transfer rate is defined as,
Q = U×A×LMTD
where, LMTD is the logarithmic mean temperature difference.
In this equation only the definition of mean temperature difference has been made more specific. Otherwise, the heat transfer rate is still governed by the product - UA.
Here is a list of equations governing the design of a shell & tube exchanger. You can use them to calculate U and A and the product UA. That will help you with selection of the best heat exchanger for your project.