I am using GIMP.
I want to make a new layer (above another layer) with round edges. How to do it? thanks
Layers are always rectangular - not only in GIMP, but in all other raster image editing software I know off.
What could restrain the layer shape and for all effects to make it look like it does have rounded colors is to create a layer mask in full white (meaning the opacity of the layer is not changed) but for the corners of the mask, which would be painted black.
Unfortunately, there is no "shortcut" for that in GIMP - so follow the steps bellow. Keep in mind that by setting "Edit->preferences->itnerface>Dynamic keyboard shortcuts" you can assigning a single shortcut for most operations bellow. Asking someone to write you (or write yourself) a Python script to achieve the results is another wya if you are doing this a log.
With your layer selected,
Add Layer Mask.... Pick "Full opacity" (white)
Mask to Selection
Select->Feather...feather it by the desired radius.
Select->Feather...again, but this time y 1 or 2px - in order to avoid aliasing
You are done. Just remember clicking back on the layer thumbnail on the Layer's dialog to ensure you are now editing the layer,instead of the mask.