Urlsession on A+ programming moments

Alley, automatic retries for any URLSessionDataTask

Wed, 11 Dec 2019 18:06:40 +0000

On my perpetual path of removing the oft-repeated boilerplate from each project I tackle, I’ve built a really small wrapper around URLSessionDataTask to implement one feature that’s always requested at some point: automatic request retries.

It’s called Alley and I encourage you to just take Alley folder and copy it into your project. Then adjust as you see fit.

In most cases where you need to fetch something from the internet, you:

Want to get the data at the URL you are targeting, no matter what.
In case when it’s simply not possible, display some useful error to the end-customer and display / log what error actually happened so you can troubleshoot and debug.

Second point is merely nice to have, although a very nice to have. First one though, that’s the main deal: that data is the reason you are performing this request at all. So in just about any client project, automatic retry appears as task to do at some point.

The main design goal of this wrapper: do not reinvent URLSession. I have no desire to write (nor use) yet another network library where you need to setup a bunch of stuff or re-factor your networking code in order to make use of it.

I assume that you would already have some URLSession instance to handle URLRequests. Then instead of this:

let urlRequest = URLRequest(...)

urlSession.dataTask(with: urlRequest) {
	data, urlResponse, error in
	//...process error, response, data
}

task.resume()

with Alley you will do this:

let urlRequest = URLRequest(...)

urlSession.perform(urlRequest, maxRetries: 5) {
	dataResult in
	//...process dataResult
}

That closure with dataResult will be called after up to 5 attempts were made to execute the URLRequest. In the current version retries will happen if URLSessionDataTask returns one of these URLError instances:

.timedOut
.cannotFindHost
.cannotConnectToHost
.networkConnectionLost
.dnsLookupFailed

I plan to update this list as I find new appropriate cases for it.

If maxRetries argument is not sent, it will fallback to default value of 10, declared on URLSession as static property maximumNumberOfRetries.

The README for the repository provides detailed explanation for DataResult and NetworkError types, thus I won’t repeat that here.

Core of the functionality is just one file, which adds some useful methods to URLSession. The only one you will use is the perform() mentioned above. Internally, it’s pretty straightforward:

perform → authenticate → execute → (process) → validate

and then either return to caller by executing the Callback provided in perform() or go back to authenticate to try again.

authenticate method seems superfluous and strangely named but I did it on purpose to make a suggestion where you can hook-up OAuth2 or some other authorization.

Alley is really simple but ready to build complete web API client libraries by simply extending what’s already there.

It’s not at 1.0 yet because I want to test it a bit more, refine if needed and maybe see if I can make it usable as Swift Package or Carthage or CocoaPods. Although I’ll probably never use it in such a way – I’ve built it with the same concept in mind as Fields form-building library: a solid foundation for concrete implementation of service wrappers.

URLSession and Operation, sitting in a tree

Tue, 31 Oct 2017 10:49:40 +0000

I don’t choose lightly to go against Apple recommendations on how to use their frameworks thus I spent quite a bit of time pondering the conundrum explained in the last post.

Let me try to paint a picture of the problem…

On one hand I have this chain of values where || indicates that URLSessionDataTask is private property populated with value created inside Operation.

APIwrapper
	⤷ URLSession
	⤷ OperationQueue
			⤷ Operation
					⤷ NetworkPayload
					⤷ URLSession, |URLSessionDataTask|

On the other hand, I have URLSessionDelegate working like this:

APIwrapper                ↰
⇣	⤷ URLSession            ⇡
⇣		⤷ URLSessionDelegate ⤴︎
⇣						⇣
⤷		func urlSession(..., dataTask: URLSessionDataTask...)

This one instance of URLSession is kept on API client wrapper, it’s passed into each Operation but all the data I get through it are received in a completely different chain of objects.

The only unique bridge between the delegate callbacks chain and the Operation is the URLSessionDataTask instance. It’s the very same instance in both chains. Thus I need to use it as bridge: take the data received in the URLSessionDelegate callbacks and attach them to the Task instance there; later read them inside the Operation.

This is what I end up with:

Extend URLSessionTask with a bunch of closure properties
Closures are set inside the Operation, so they have access to its scope and thus to NetworkPayload
Those closures will receive input from URLSessionDelegate callbacks, passing in the received data

Again and again, I am very, very grateful to whoever added associated objects to Objective-C runtime, since it allows me to overcome Swift’s (current) inability to declare stored properties in type extension:

extension URLSessionTask {
	public typealias NetworkTaskErrorCallback = (NetworkError) -> Void
	...

	private struct AssociatedKeys {
		static var error 	= "Error"
		...
	}

	public var errorCallback: NetworkTaskErrorCallback {
		get {
			return objc_getAssociatedObject(self, &AssociatedKeys.error) as? NetworkTaskErrorCallback ?? {_ in}
		}
		set {
			objc_setAssociatedObject(self, &AssociatedKeys.error, newValue, .OBJC_ASSOCIATION_RETAIN)
		}
	}

	...
}

Thus inside NetworkOperation I will declare the closure(s):

func setupCallbacks() {
	guard let task = task else { return }

	task.errorCallback = {
		[weak self] error in
		self?.payload.error = error
		self?.finish()
	}
	...
}

and execute them from the appropriate URLSessionDelegate callback:

final func urlSession(_ session: URLSession,
					  task: URLSessionTask,
					  didCompleteWithError error: Swift.Error?) {
	guard 
		let dataTask = task as? URLSessionDataTask 
	else { return }

	if let e = error {
		dataTask.errorCallback( .urlError(e as? URLError) )
		return
	}
	dataTask.finishCallback()
}

I now have a fully working solution that allows me to use Operations for network requests, while still adhering to Apple’s recommendations of (generally) using one URLSession per app.

Thoughts on URLSession architecture

Thu, 19 Oct 2017 23:09:40 +0000

In my talk / post about LAYERS architecture I specifically pointed out how I deal with networking: I have downgraded it to pure, self-contained one-off utility.

In short – I have an Operation subclass which receives URLRequest + URLSessionConfiguration as input, creates an instance of URLSession inside the Op, execute the request and it all goes away once result is delivered. Each network request gets its own URLSession, 1 to 1.

This turned out to be the most controversial part; I received few direct and a bit more indirect feedback about this. Using URLSession in such a way is exactly what Apple recommends not to do (jump to ~ 32:40):

I know this and for the longest time I was going with one URLSession story in all my apps. Heck, it was trivial to update my current networking micro-library to allow such usage.

On the other hand, I do like Operations a lot. They give me so much flexibility, especially when I need to temporary pause the stream of requests to either fetch some session cookie or new OAuth access token. While that’s ongoing, all the incoming requests can easily be queued-up in the OperationQueue and later authenticated as needed.

Here’s the rub: the intricate way URLSession / URLSessionDataTask API is designed does not work well with Operations. That’s not obvious if you use the most popular form:

let task = localURLSession.dataTask(with: urlRequest, completionHandler: {
	[weak self] data, response, error in
	...
})

because that can be wrapped into Operation, very easily.

With huge caveat: it works as long as you don’t need to handle any sort of AuthenticationChallenge, say if your development API server is using self-signed SSL certificate. In which case completionHandler form is useless since you need to use URLSessionDelegate form:

let localURLSession = URLSession(configuration: urlSessionConfiguration, delegate: self, delegateQueue: nil)

let task = localURLSession.dataTask(with: urlRequest)

in order to handle the challenge. The very next slide in the same talk from WWDC17 mentions that particular fact:

Look at the last part: URLSession will still make authentication-related delegate callbacks even if you use completionHandler form for the DataTask. It does do that but it’s totally useless, because no matter what you do in the delegate callback, your completionHandler will always receive the same result, this SSL error:

Printing description of e._userInfo:
{
    NSErrorClientCertificateStateKey = 0;
    NSErrorFailingURLKey = "https://HOST/PATH";
    NSErrorFailingURLStringKey = "https://HOST/PATH";
    NSErrorPeerCertificateChainKey =     (
        "<cert(0x7fcbb48c1000) s: *.OTHERHOST.com i: RapidSSL SHA256 CA - G2>",
        "<cert(0x7fcbb48c1800) s: RapidSSL SHA256 CA - G2 i: GeoTrust Primary Certification Authority - G3>",
        "<cert(0x7fcbb48bd000) s: GeoTrust Primary Certification Authority - G3 i: GeoTrust Primary Certification Authority - G3>"
    );
    NSLocalizedDescription = "An SSL error has occurred and a secure connection to the server cannot be made.";
    NSLocalizedRecoverySuggestion = "Would you like to connect to the server anyway?";
    NSURLErrorFailingURLPeerTrustErrorKey = "<SecTrustRef: 0x608000109b40>";
    NSUnderlyingError = "Error Domain=kCFErrorDomainCFNetwork Code=-1200 \"(null)\" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, kCFStreamPropertySSLPeerTrust=<SecTrustRef: 0x608000109b40>, _kCFNetworkCFStreamSSLErrorOriginalValue=-9802, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9802, kCFStreamPropertySSLPeerCertificates=(\n    \"<cert(0x7fcbb48c1000) s: *.OTHERHOST.com i: RapidSSL SHA256 CA - G2>\",\n    \"<cert(0x7fcbb48c1800) s: RapidSSL SHA256 CA - G2 i: GeoTrust Primary Certification Authority - G3>\",\n    \"<cert(0x7fcbb48bd000) s: GeoTrust Primary Certification Authority - G3 i: GeoTrust Primary Certification Authority - G3>\"\n)}";
    "_kCFStreamErrorCodeKey" = "-9802";
    "_kCFStreamErrorDomainKey" = 3;
}

If I do this in the callback:

func urlSession(_ session: URLSession, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void) {

	if challenge.protectionSpace.authenticationMethod == NSURLAuthenticationMethodServerTrust {
		let trust = challenge.protectionSpace.serverTrust!
		let host = challenge.protectionSpace.host
		guard session.serverTrustPolicy.evaluate(trust, forHost: host) else {
			completionHandler(.rejectProtectionSpace, nil)
			return
		}

		let credential = URLCredential(trust: trust)
		completionHandler(.useCredential, credential)
		return
	}

	completionHandler(.performDefaultHandling, nil)
}

I expect it to accept my decision and trust the server, not give me the error. Otherwise what’s the point of making the callback then?

(This is the moment I hope someone will point out some glaring mistake I made here…)

UPDATE, Oct 23rd

Correct ATS incantation in Info.plist is the answer here, I have no idea why my previous attempts at it did not work. Here’s what I added in this test project so that completionHandler(.useCredential, credential) above works as expected:

<key>NSAppTransportSecurity</key>
<dict>
	<key>NSExceptionDomains</key>
	<dict>
		<key>self-signed.badssl.com</key>
		<dict>
			<key>NSExceptionAllowsInsecureHTTPLoads</key>
			<true/>
		</dict>
	</dict>
</dict>

Or in Xcode:

Going full-on URLSessionDelegate

Ok, let’s go entirely with delegate form and implement all of the callbacks. This is perfectly possible only if both URLSession instance and URLSessionDataTask instance are inside the Operation subclass which is acting as URLSessionDelegate.

Back in the Apple corner, they say I need to have one URLSession instance, which means it must be outside the Operation. However, once URLSession.delegate is assigned to something – that’s final. I can’t change it from one Operation to another. Understandably so since there’s no way to control where incoming data chunks end up if various Operations start stealing the delegate.

So the only solution seems to be: forget about concurrent network requests and execute them one by one. Does not sound right to me.

What I don’t grasp here is why there’s no such thing as URLSessionTask.delegate which will handle task specific stuff, like receiving data, response headers etc, for that specific request.

Also: if there should be one instance which handles session-level Auth challenges – like server trust – why then keep asking me to handle for each task in that same session. All the tasks are using the very same server. Honestly, I can imagine some MitM attack vector here so I sort of understand why they ask each time. But then – what’s the point on having the delegate on the URLSession level and not on the URLSessionTask?

The API design forces me to create some higher-level object to act as URLSessionDelegate which then forces me to build some convoluted structure of keeping track which URLSessionDataTask are active so I can route data received here:

func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) {}

func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {}

into proper DataTask instances I keep…somewhere…

I don’t understand how to do concurrent network requests using URLSessionDelegate + replicate what OperationQueue/Operation gives me if I need to use the same session + delegate instance for all the tasks.

Ability to have totally independent network operations is way more important to me than adhering to these best practices.

UPDATE, Oct 23rd

I may have a solution here. Still need to test it in real project and see are there any drawbacks.