I have raw data like this:

COL1 COL2 VALUE
-----------------
a     x     6
a     x     3
a     y     4
a     y     8
b     x     7
b     x     30
b     x     10
b     x     20
b     z     2
b     z     1

etc.

I need to make a report with Grouping by COL1, COL2 and with parameter field (%).
I'll try to explain what this field means by example.

Let's assume that when I run report I set this field to 50%. It means that I need to calculate maximum value for EACH group taking into account 50% or raw data which sorted by VALUE ascending.
In other words:
1) GROUPING BY COL2
1a) for "a x" 2 rows, 2*0.5=1 - I need maximum of 1 smallest rows
1b) for "a y" 2 rows, 2*0.5=1 - I need maximum of 1 smallest rows
1c) for "b x" 4 rows, 4*0.5=2 - I need maximum of 2 smallest rows
1d) for "b z" 2 rows, 2*0.5=1 - I need maximum of 1 smallest rows
2) GROUPING BY COL1
2a) for "a" 4 rows, 4*0.5=2 - I need maximum of 2 smallest rows
2b) for "b" 6 rows, 6*0.5=3 - I need maximum of 3 smallest rows
3) GRAND TOTAL
10 rows, 10*0.5=5 - I need maximum of 5 smallest rows

Result
------
a      4   
    x 3
    y 4
b      7
    x 10
    z 1
Total: 6

I can create such report with static N by using standard tool "Add Summary -> Nth smallest ,N is: -> enter N", but it does not allow to enter a formula or things like that.
Please help to find a solution.
Many thanks in advance for any advice!