At Clever Cloud, we use a lot of Java. A large part of our API is written using JEE (aka Java Enterprise Edition). Here is how we implemented the http client we use for our Github integration.
The standard HTTP implementation in JEE is Jersey, so that's what we'll use here.
A bit of background first
Our API is designed in a "micro-services" way. There's a central part and various external components.
Here, we'll be dealing with the central part, which itself is internally designed in a sort-of "micro-services" way. JEE offers a powerful dependency injection mechanism called CDI, which we use to build a lot of small tools we can then use just as we need.
Serialization
When you're building APIs and clients, one of the problematics is about serialization. We made a tool to do just that which you'll see in the code as MapperHelper, namely mh. It's basically a wrapper around jackson's ObjectMapper.
Let's get to work
We'll be working on an internal tool called GithubAPIHelper. We'll not get through the whole of it, but you'll quickly get the concept.
First, some standard JEE boilerplate injecting required tools using CDI.
@Stateless
public class GithubAPIHelper {
@Inject
private MapperHelper mh;
@Inject
private LogsHelper lh;
private final static String GITHUB_URL = "https://api.github.com";
}
Now, we want to build an HTTP client using jersey-client, this is pretty straightforward. Btw, we create a method to make the client directly point at Github.
private Client getClient() {
if (client == null) {
client = ClientBuilder.newClient(new ClientConfig().property(ClientProperties.FOLLOW_REDIRECTS, true));
}
return client;
}
private WebTarget getTarget() {
return getClient().target(GITHUB_URL);
}
What about authentication? Let's create a new method to get an authenticated request to Github:
private Invocation.Builder getBuilder(WebTarget target, String token) {
return target.request().header("Authorization", "token " + token);
}
Ok, now that we have those basic pieces in place, we can start requesting for real.
Let's get the basic user profile, for starters
First, we create a basic data structure that we'll use to deserialize the payload from Github. I'll skip most of the available fields here as they're not needed.
public class GithubOwner implements Serializable {
public int id;
public String name;
public String login;
public String email;
public String avatar_url;
}
Now, you should all be yelling at me about those public fields. Let me remember you that this data structure is only used for deserializing purpose, it won't ever be used for anything else, so what's the point in complicating things?
Now let's get back to business, we have a way to construct authenticated requests, and a model in which to store the data. So let's do it!
First, let's get to the right endpoint: /user
getTarget().path("/user");
Now, let's actually issue the request, and get a response
Response r = getBuilder(getTarget().path("/user"), token).get();
Let's deserialize the data from github
GithubOwner owner = mh.readValue(r.readEntity(String.class), GithubOwner.class);
And here we are, with some integrity checks added:
public GithubOwner getSelf(String token) {
Response r = getBuilder(getTarget().path("/user"), token).get();
if (r.getStatus() < 300) {
try {
GithubOwner owner = mh.readValue(r.readEntity(String.class), GithubOwner.class);
return owner;
} catch (IOException e) {
lh.log(GithubAPIHelper.class.getName(), Level.SEVERE, null, e);
}
}
return null;
}
Dealing with pagination
Getting the user's profile was quite easy and straightforward, but everything isn't that smooth.
Let's say we now want all the repositories, instinctively, here is what we'd do:
public class GithubRepository implements Serializable {
public long id;
public String name;
public String description;
public GithubOwner owner;
@JsonProperty(value = "private")
public boolean isPrivate;
public String ssh_url;
public String git_url;
}
private TypeSafeList<GithubRepository> getSelfRepositories(String token) {
TypeSafeList<GithubRepository> repos = new TypeSafeArrayList<>();
Response r = getBuilder(getTarget().path("/user/repos"), token).get();
if (r.getStatus() < 300) {
try {
List<GithubRepository> ms = mh.readValue(r.readEntity(String.class), new TypeReference<List<GithubRepository>>() {
});
repos.addAll(new TypeSafeArrayList<>(ms));
} catch (IOException e) {
lh.log(GithubAPIHelper.class.getName(), Level.SEVERE, null, e);
}
}
return repos;
}
That will actually work, but you won't get all of them, only the first 20 or so. If you want all of them, you'll have to follow the pagination.
First, we want to get the link marked as rel=next in the Link http header.
private String getNextLink(Response r) {
String link = r.getHeaderString("Link"); /* <http://foobar>; rel="next", <http://blah/>; rel=last */
if (link != null) {
String[] links = link.split(",");
for (String l : links) { /* <http://foobar>; rel="next" */
if (l.contains("rel=\"next\"")) {
String[] tmp1 = l.split("<", 2);
if (tmp1.length == 2) {
return tmp1[1].split(">", 2)[0]; /* http://foobar */
}
}
}
}
return null;
}
Next, we want to request this url and get the next data to our initial request
private Response getNextData(Response r, String token) {
String link = getNextLink(r);
if (link == null)
return null;
return getBuilder(getClient().target(link), token).get();
}
Now let's put all the pieces together, addind a loop to retrieve all the data
private TypeSafeList<GithubRepository> getSelfRepositories(String token) {
TypeSafeList<GithubRepository> repos = new TypeSafeArrayList<>();
Response r = getBuilder(getTarget().path("/user/repos"), token).get();
while (r != null) {
if (r.getStatus() < 300) {
try {
List<GithubRepository> ms = mh.readValue(r.readEntity(String.class), newhTypeReference<List<GithubRepository>>() {
});
repos.addAll(new TypeSafeArrayList<>(ms));
} catch (IOException e) {
lh.log(GithubAPIHelper.class.getName(), Level.SEVERE, null, e);
}
}
r = getNextData(r, token);
}
return repos;
}
And voila, you now have a working Github HTTP client. Once you got these, the other endpoints are really easy.